2024-04-26T19:42:55.756 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T19:42:55.756 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/snap_schedule_snapdir} --verbose --name rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111 -- /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/orig.config.yaml 2024-04-26T19:42:55.803 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T19:42:55.890 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111 branch: wip-rishabh-testing-20240426.111959 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/snap_schedule_snapdir} email: null first_in_suite: false job_id: '7675111' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240426.111959 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 client snapdir: .customsnapfuse debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 kclient: snapdirname: .customsnapkernel syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240426.111959 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 816 sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 sleep_before_teardown: 0 subset: 166/512 suite: fs suite_branch: wip-rishabh-testing-20240426.111959 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi079.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzJLScmthQdf3TjaKOuZ2+yARE0Zi4vrwZkN45ntwzkqt8/gyGQVAFu5Ss00qohimKKK1AiZtzvbB3w3rO8Euo= smithi083.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1c0p100gtm5EsmevwkWsnBpyVPa5K13Ky32wKej6fx0tGghVSxbcOOS7/YKJznYeffvlzJSUrfHO7SN73k99Y= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_19:30:57 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T19:42:55.891 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa; will attempt to use it 2024-04-26T19:42:55.891 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa/tasks 2024-04-26T19:42:55.891 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T19:42:55.893 INFO:teuthology.task.internal:Checking packages... 2024-04-26T19:42:55.916 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc612ffb9c1d5bc27536bc917a705262a36d4387' 2024-04-26T19:42:55.917 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T19:42:55.917 INFO:teuthology.packaging:ref: None 2024-04-26T19:42:55.917 INFO:teuthology.packaging:tag: None 2024-04-26T19:42:55.917 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240426.111959 2024-04-26T19:42:55.917 INFO:teuthology.packaging:sha1: bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:42:55.917 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240426.111959 2024-04-26T19:42:56.140 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3306.gbc612ffb 2024-04-26T19:42:56.141 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T19:42:56.147 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T19:42:56.147 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T19:42:56.182 INFO:teuthology.task.internal:Saving configuration 2024-04-26T19:42:56.200 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T19:42:56.207 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T19:42:56.230 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi079.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 19:36:39.352138', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzJLScmthQdf3TjaKOuZ2+yARE0Zi4vrwZkN45ntwzkqt8/gyGQVAFu5Ss00qohimKKK1AiZtzvbB3w3rO8Euo='} 2024-04-26T19:42:56.248 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi083.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 19:36:39.353955', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1c0p100gtm5EsmevwkWsnBpyVPa5K13Ky32wKej6fx0tGghVSxbcOOS7/YKJznYeffvlzJSUrfHO7SN73k99Y='} 2024-04-26T19:42:56.248 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T19:42:56.255 INFO:teuthology.task.internal:roles: ubuntu@smithi079.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-26T19:42:56.255 INFO:teuthology.task.internal:roles: ubuntu@smithi083.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-26T19:42:56.255 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T19:42:56.429 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f87a3d70160>, signals=[15]) 2024-04-26T19:42:56.430 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T19:42:56.440 INFO:teuthology.task.internal:Opening connections... 2024-04-26T19:42:56.441 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:42:56.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:42:56.523 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:42:56.524 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:42:56.598 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T19:42:56.606 DEBUG:teuthology.orchestra.run.smithi079:> uname -m 2024-04-26T19:42:56.623 INFO:teuthology.orchestra.run.smithi079.stdout:x86_64 2024-04-26T19:42:56.623 DEBUG:teuthology.orchestra.run.smithi079:> cat /etc/os-release 2024-04-26T19:42:56.683 INFO:teuthology.orchestra.run.smithi079.stdout:NAME="CentOS Stream" 2024-04-26T19:42:56.683 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION="9" 2024-04-26T19:42:56.683 INFO:teuthology.orchestra.run.smithi079.stdout:ID="centos" 2024-04-26T19:42:56.683 INFO:teuthology.orchestra.run.smithi079.stdout:ID_LIKE="rhel fedora" 2024-04-26T19:42:56.683 INFO:teuthology.orchestra.run.smithi079.stdout:VERSION_ID="9" 2024-04-26T19:42:56.683 INFO:teuthology.orchestra.run.smithi079.stdout:PLATFORM_ID="platform:el9" 2024-04-26T19:42:56.683 INFO:teuthology.orchestra.run.smithi079.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T19:42:56.683 INFO:teuthology.orchestra.run.smithi079.stdout:ANSI_COLOR="0;31" 2024-04-26T19:42:56.683 INFO:teuthology.orchestra.run.smithi079.stdout:LOGO="fedora-logo-icon" 2024-04-26T19:42:56.684 INFO:teuthology.orchestra.run.smithi079.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T19:42:56.684 INFO:teuthology.orchestra.run.smithi079.stdout:HOME_URL="https://centos.org/" 2024-04-26T19:42:56.684 INFO:teuthology.orchestra.run.smithi079.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T19:42:56.684 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T19:42:56.684 INFO:teuthology.orchestra.run.smithi079.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T19:42:56.684 INFO:teuthology.lock.ops:Updating smithi079.front.sepia.ceph.com on lock server 2024-04-26T19:42:56.703 DEBUG:teuthology.orchestra.run.smithi083:> uname -m 2024-04-26T19:42:56.721 INFO:teuthology.orchestra.run.smithi083.stdout:x86_64 2024-04-26T19:42:56.722 DEBUG:teuthology.orchestra.run.smithi083:> cat /etc/os-release 2024-04-26T19:42:56.778 INFO:teuthology.orchestra.run.smithi083.stdout:NAME="CentOS Stream" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION="9" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:ID="centos" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:ID_LIKE="rhel fedora" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:VERSION_ID="9" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:PLATFORM_ID="platform:el9" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:ANSI_COLOR="0;31" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:LOGO="fedora-logo-icon" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:HOME_URL="https://centos.org/" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T19:42:56.779 INFO:teuthology.orchestra.run.smithi083.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T19:42:56.780 INFO:teuthology.lock.ops:Updating smithi083.front.sepia.ceph.com on lock server 2024-04-26T19:42:56.795 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T19:42:56.804 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T19:42:56.829 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T19:42:56.830 DEBUG:teuthology.orchestra.run.smithi079:> test '!' -e /home/ubuntu/cephtest 2024-04-26T19:42:56.832 DEBUG:teuthology.orchestra.run.smithi083:> test '!' -e /home/ubuntu/cephtest 2024-04-26T19:42:56.848 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T19:42:56.909 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T19:42:56.909 DEBUG:teuthology.orchestra.run.smithi079:> test -z $(ls -A /var/lib/ceph) 2024-04-26T19:42:56.912 DEBUG:teuthology.orchestra.run.smithi083:> test -z $(ls -A /var/lib/ceph) 2024-04-26T19:42:57.020 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T19:42:57.080 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T19:42:57.095 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-26T19:42:57.096 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2024-04-26T19:42:57.096 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-04-26T19:42:57.096 DEBUG:teuthology.orchestra.run.smithi079:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T19:42:57.096 DEBUG:teuthology.orchestra.run.smithi083:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T19:42:57.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:42:57.113 DEBUG:teuthology.orchestra.run.smithi079:> uname -r 2024-04-26T19:42:57.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:42:57.114 DEBUG:teuthology.orchestra.run.smithi083:> uname -r 2024-04-26T19:42:57.170 INFO:teuthology.orchestra.run.smithi079.stdout:5.14.0-437.el9.x86_64 2024-04-26T19:42:57.170 INFO:teuthology.task.kernel:Running kernel on smithi079: 5.14.0-437.el9.x86_64 2024-04-26T19:42:57.170 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum install -y kernel 2024-04-26T19:42:57.171 INFO:teuthology.orchestra.run.smithi083.stdout:5.14.0-437.el9.x86_64 2024-04-26T19:42:57.171 INFO:teuthology.task.kernel:Running kernel on smithi083: 5.14.0-437.el9.x86_64 2024-04-26T19:42:57.171 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum install -y kernel 2024-04-26T19:43:07.959 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 902 kB/s | 8.0 MB 00:09 2024-04-26T19:43:08.100 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 895 kB/s | 8.0 MB 00:09 2024-04-26T19:43:11.461 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 19 MB 00:02 2024-04-26T19:43:13.067 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 4.7 MB/s | 19 MB 00:04 2024-04-26T19:43:15.568 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-26T19:43:17.426 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.2 MB 00:00 2024-04-26T19:43:17.719 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 16 kB 00:00 2024-04-26T19:43:19.110 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T19:43:19.864 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-26T19:43:20.552 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-26T19:43:25.062 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-04-26T19:43:25.518 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T19:43:26.926 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T19:43:26.927 INFO:teuthology.orchestra.run.smithi083.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T19:43:27.007 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-26T19:43:27.008 INFO:teuthology.orchestra.run.smithi083.stdout:Nothing to do. 2024-04-26T19:43:27.008 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-26T19:43:27.103 DEBUG:teuthology.orchestra.run.smithi083:> echo no | sudo yum reinstall kernel || true 2024-04-26T19:43:27.425 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T19:43:27.425 INFO:teuthology.orchestra.run.smithi079.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T19:43:27.517 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-04-26T19:43:27.518 INFO:teuthology.orchestra.run.smithi079.stdout:Nothing to do. 2024-04-26T19:43:27.519 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-04-26T19:43:27.601 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 07:43:25 PM UTC. 2024-04-26T19:43:27.627 DEBUG:teuthology.orchestra.run.smithi079:> echo no | sudo yum reinstall kernel || true 2024-04-26T19:43:27.721 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-26T19:43:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-26T19:43:27.732 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-04-26T19:43:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-26T19:43:27.732 INFO:teuthology.orchestra.run.smithi083.stdout:Reinstalling: 2024-04-26T19:43:27.732 INFO:teuthology.orchestra.run.smithi083.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T19:43:27.732 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:43:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-26T19:43:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-26T19:43:27.733 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:43:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 255 k 2024-04-26T19:43:27.733 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 0 2024-04-26T19:43:27.733 INFO:teuthology.orchestra.run.smithi083.stderr:Operation aborted. 2024-04-26T19:43:27.792 INFO:teuthology.orchestra.run.smithi083.stdout:Is this ok [y/N]: 2024-04-26T19:43:27.793 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum reinstall -y kernel || true 2024-04-26T19:43:28.124 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 07:43:25 PM UTC. 2024-04-26T19:43:28.253 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-04-26T19:43:28.263 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-04-26T19:43:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-04-26T19:43:28.263 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-04-26T19:43:28.263 INFO:teuthology.orchestra.run.smithi079.stdout:Reinstalling: 2024-04-26T19:43:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T19:43:28.263 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:43:28.263 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-04-26T19:43:28.264 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-04-26T19:43:28.264 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:43:28.264 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 255 k 2024-04-26T19:43:28.264 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 0 2024-04-26T19:43:28.265 INFO:teuthology.orchestra.run.smithi079.stderr:Operation aborted. 2024-04-26T19:43:28.288 INFO:teuthology.orchestra.run.smithi083.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 07:43:25 PM UTC. 2024-04-26T19:43:28.321 INFO:teuthology.orchestra.run.smithi079.stdout:Is this ok [y/N]: 2024-04-26T19:43:28.329 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum reinstall -y kernel || true 2024-04-26T19:43:28.406 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-26T19:43:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-26T19:43:28.407 INFO:teuthology.orchestra.run.smithi083.stdout: Package Architecture Version Repository Size 2024-04-26T19:43:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-26T19:43:28.407 INFO:teuthology.orchestra.run.smithi083.stdout:Reinstalling: 2024-04-26T19:43:28.407 INFO:teuthology.orchestra.run.smithi083.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T19:43:28.407 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:43:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-26T19:43:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:================================================================================ 2024-04-26T19:43:28.408 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:43:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 255 k 2024-04-26T19:43:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:Installed size: 0 2024-04-26T19:43:28.408 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-26T19:43:28.821 INFO:teuthology.orchestra.run.smithi079.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 07:43:25 PM UTC. 2024-04-26T19:43:28.864 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-437.el9.x86_64.rpm 867 kB/s | 255 kB 00:00 2024-04-26T19:43:28.864 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:43:28.864 INFO:teuthology.orchestra.run.smithi083.stdout:Total 558 kB/s | 255 kB 00:00 2024-04-26T19:43:28.880 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-26T19:43:28.936 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: Package Architecture Version Repository Size 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout:Reinstalling: 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout:================================================================================ 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:43:28.937 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 255 k 2024-04-26T19:43:28.938 INFO:teuthology.orchestra.run.smithi079.stdout:Installed size: 0 2024-04-26T19:43:28.938 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-04-26T19:43:29.071 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-26T19:43:29.071 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-26T19:43:29.076 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-26T19:43:29.076 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-26T19:43:29.206 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-26T19:43:29.265 INFO:teuthology.orchestra.run.smithi083.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T19:43:29.265 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T19:43:29.362 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.7 MB/s | 255 kB 00:00 2024-04-26T19:43:29.362 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:43:29.363 INFO:teuthology.orchestra.run.smithi079.stdout:Total 600 kB/s | 255 kB 00:00 2024-04-26T19:43:29.385 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-04-26T19:43:29.568 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-04-26T19:43:29.568 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-04-26T19:43:29.573 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-04-26T19:43:29.573 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-04-26T19:43:29.696 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-04-26T19:43:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T19:43:29.780 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T19:43:29.934 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T19:43:29.934 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T19:43:30.440 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T19:43:30.440 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T19:43:30.610 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T19:43:30.610 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:43:30.610 INFO:teuthology.orchestra.run.smithi083.stdout:Reinstalled: 2024-04-26T19:43:30.610 INFO:teuthology.orchestra.run.smithi083.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T19:43:30.610 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:43:30.610 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-26T19:43:30.780 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T19:43:30.826 INFO:teuthology.orchestra.run.smithi083.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T19:43:30.827 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T19:43:30.827 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T19:43:30.827 DEBUG:teuthology.parallel:result is None 2024-04-26T19:43:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T19:43:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:43:31.136 INFO:teuthology.orchestra.run.smithi079.stdout:Reinstalled: 2024-04-26T19:43:31.136 INFO:teuthology.orchestra.run.smithi079.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T19:43:31.137 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:43:31.137 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-04-26T19:43:31.313 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T19:43:31.359 INFO:teuthology.orchestra.run.smithi079.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T19:43:31.360 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T19:43:31.360 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T19:43:31.360 DEBUG:teuthology.parallel:result is None 2024-04-26T19:43:31.360 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T19:43:31.367 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T19:43:31.367 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T19:43:31.369 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T19:43:31.386 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T19:43:31.392 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T19:43:31.400 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T19:43:31.400 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T19:43:31.424 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T19:43:31.460 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T19:43:31.466 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T19:43:31.466 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T19:43:31.487 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T19:43:31.522 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:43:31.526 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:43:31.533 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:43:31.560 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T19:43:31.561 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T19:43:31.568 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T19:43:31.569 DEBUG:teuthology.orchestra.run.smithi079:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T19:43:31.578 DEBUG:teuthology.orchestra.run.smithi083:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T19:43:31.627 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T19:43:31.636 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T19:43:31.636 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T19:43:31.642 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T19:43:31.683 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:43:31.734 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:43:31.793 DEBUG:teuthology.orchestra.run.smithi079:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:43:31.875 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:43:31.914 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:43:31.914 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T19:43:31.976 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:43:32.017 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T19:43:32.070 DEBUG:teuthology.orchestra.run.smithi083:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:43:32.151 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T19:43:32.194 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:43:32.194 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T19:43:32.258 DEBUG:teuthology.orchestra.run.smithi079:> sudo service rsyslog restart 2024-04-26T19:43:32.261 DEBUG:teuthology.orchestra.run.smithi083:> sudo service rsyslog restart 2024-04-26T19:43:32.291 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T19:43:32.340 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T19:43:32.850 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T19:43:32.857 INFO:teuthology.task.internal:Starting timer... 2024-04-26T19:43:32.858 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T19:43:32.868 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T19:43:32.875 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T19:43:32.876 DEBUG:teuthology.orchestra.run.smithi079:> sudo service auditd rotate 2024-04-26T19:43:32.919 INFO:teuthology.orchestra.run.smithi079.stdout:Rotating logs: 2024-04-26T19:43:32.920 DEBUG:teuthology.orchestra.run.smithi083:> sudo service auditd rotate 2024-04-26T19:43:32.979 INFO:teuthology.orchestra.run.smithi083.stdout:Rotating logs: 2024-04-26T19:43:32.981 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T19:43:32.981 DEBUG:teuthology.orchestra.run.smithi079:> /usr/sbin/getenforce 2024-04-26T19:43:33.001 INFO:teuthology.orchestra.run.smithi079.stdout:Permissive 2024-04-26T19:43:33.001 DEBUG:teuthology.orchestra.run.smithi083:> /usr/sbin/getenforce 2024-04-26T19:43:33.021 INFO:teuthology.orchestra.run.smithi083.stdout:Permissive 2024-04-26T19:43:33.021 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi079.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi083.front.sepia.ceph.com': 'permissive'} 2024-04-26T19:43:33.021 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T19:43:33.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:43:33.065 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T19:43:33.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:43:33.089 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T19:43:33.089 DEBUG:teuthology.orchestra.run.smithi079:> sudo /usr/sbin/setenforce permissive 2024-04-26T19:43:33.133 DEBUG:teuthology.orchestra.run.smithi083:> sudo /usr/sbin/setenforce permissive 2024-04-26T19:43:33.158 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T19:43:33.168 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T19:43:33.173 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T19:43:33.298 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T19:43:33.313 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T19:43:33.314 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi079.front.sepia.ceph.com,smithi083.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T19:47:00.851 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi079.front.sepia.ceph.com'), Remote(name='ubuntu@smithi083.front.sepia.ceph.com')] 2024-04-26T19:47:00.852 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-04-26T19:47:00.853 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi079.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:47:00.932 DEBUG:teuthology.orchestra.run.smithi079:> true 2024-04-26T19:47:01.012 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi079.front.sepia.ceph.com' 2024-04-26T19:47:01.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-26T19:47:01.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi083.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T19:47:01.088 DEBUG:teuthology.orchestra.run.smithi083:> true 2024-04-26T19:47:01.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi083.front.sepia.ceph.com' 2024-04-26T19:47:01.171 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T19:47:01.182 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T19:47:01.182 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T19:47:01.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T19:47:01.185 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T19:47:01.185 DEBUG:teuthology.orchestra.run.smithi083:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T19:47:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T19:47:01.223 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T19:47:01.242 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T19:47:01.245 INFO:teuthology.orchestra.run.smithi079.stderr:sudo: ntpd: command not found 2024-04-26T19:47:01.255 INFO:teuthology.orchestra.run.smithi079.stdout:506 Cannot talk to daemon 2024-04-26T19:47:01.256 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T19:47:01.268 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T19:47:01.279 INFO:teuthology.orchestra.run.smithi083.stderr:sudo: ntpd: command not found 2024-04-26T19:47:01.280 INFO:teuthology.orchestra.run.smithi079.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T19:47:01.289 INFO:teuthology.orchestra.run.smithi083.stdout:506 Cannot talk to daemon 2024-04-26T19:47:01.303 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T19:47:01.316 INFO:teuthology.orchestra.run.smithi083.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T19:47:01.335 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-04-26T19:47:01.338 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T19:47:01.338 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-04-26T19:47:01.338 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:47:01.338 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:47:01.338 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:47:01.338 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:47:01.374 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-26T19:47:01.377 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T19:47:01.377 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-26T19:47:01.377 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:47:01.377 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:47:01.377 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:47:01.377 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:47:01.378 INFO:teuthology.run_tasks:Running task install... 2024-04-26T19:47:01.387 DEBUG:teuthology.task.install:project ceph 2024-04-26T19:47:01.387 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}} 2024-04-26T19:47:01.387 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'} 2024-04-26T19:47:01.387 INFO:teuthology.task.install:Using flavor: default 2024-04-26T19:47:01.393 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T19:47:01.393 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T19:47:01.393 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T19:47:01.394 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:47:01.396 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'tag': None, 'wait_for_package': False} 2024-04-26T19:47:01.396 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:47:01.536 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T19:47:01.537 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T19:47:01.542 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/ 2024-04-26T19:47:01.542 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3306.gbc612ffb 2024-04-26T19:47:01.806 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T19:47:01.806 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:47:01.807 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T19:47:01.817 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240426.111959/bc612ffb9c1d5bc27536bc917a705262a36d4387/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T19:47:01.818 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:47:01.818 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T19:47:01.834 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T19:47:01.834 DEBUG:teuthology.orchestra.run.smithi079:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T19:47:01.842 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T19:47:01.842 DEBUG:teuthology.orchestra.run.smithi083:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc612ffb9c1d5bc27536bc917a705262a36d4387/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T19:47:01.900 DEBUG:teuthology.orchestra.run.smithi079:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T19:47:01.906 DEBUG:teuthology.orchestra.run.smithi083:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T19:47:01.973 DEBUG:teuthology.orchestra.run.smithi079:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T19:47:01.979 DEBUG:teuthology.orchestra.run.smithi083:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T19:47:02.038 INFO:teuthology.orchestra.run.smithi079.stdout:check_obsoletes = 1 2024-04-26T19:47:02.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum clean all 2024-04-26T19:47:02.042 INFO:teuthology.orchestra.run.smithi083.stdout:check_obsoletes = 1 2024-04-26T19:47:02.044 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum clean all 2024-04-26T19:47:02.261 INFO:teuthology.orchestra.run.smithi079.stdout:48 files removed 2024-04-26T19:47:02.267 INFO:teuthology.orchestra.run.smithi083.stdout:48 files removed 2024-04-26T19:47:02.287 DEBUG:teuthology.orchestra.run.smithi079:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T19:47:02.294 DEBUG:teuthology.orchestra.run.smithi083:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T19:47:03.215 INFO:teuthology.orchestra.run.smithi079.stdout:ceph packages for x86_64 125 kB/s | 84 kB 00:00 2024-04-26T19:47:03.233 INFO:teuthology.orchestra.run.smithi083.stdout:ceph packages for x86_64 123 kB/s | 84 kB 00:00 2024-04-26T19:47:03.709 INFO:teuthology.orchestra.run.smithi083.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-04-26T19:47:03.857 INFO:teuthology.orchestra.run.smithi079.stdout:ceph noarch packages 19 kB/s | 12 kB 00:00 2024-04-26T19:47:04.251 INFO:teuthology.orchestra.run.smithi083.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-04-26T19:47:04.325 INFO:teuthology.orchestra.run.smithi079.stdout:ceph source packages 4.4 kB/s | 1.9 kB 00:00 2024-04-26T19:47:04.875 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-26T19:47:08.245 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.0 MB 00:03 2024-04-26T19:47:10.602 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-26T19:47:15.547 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-04-26T19:47:16.887 INFO:teuthology.orchestra.run.smithi083.stdout:CentOS Stream 9 - Extras packages 78 kB/s | 16 kB 00:00 2024-04-26T19:47:20.139 INFO:teuthology.orchestra.run.smithi083.stdout:Extra Packages for Enterprise Linux 6.7 MB/s | 21 MB 00:03 2024-04-26T19:47:25.205 INFO:teuthology.orchestra.run.smithi083.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-26T19:47:25.289 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 19 MB 00:19 2024-04-26T19:47:27.315 INFO:teuthology.orchestra.run.smithi083.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:47:27.315 INFO:teuthology.orchestra.run.smithi083.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:47:27.326 INFO:teuthology.orchestra.run.smithi083.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T19:47:27.327 INFO:teuthology.orchestra.run.smithi083.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T19:47:27.327 INFO:teuthology.orchestra.run.smithi083.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T19:47:27.328 INFO:teuthology.orchestra.run.smithi083.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T19:47:27.330 INFO:teuthology.orchestra.run.smithi083.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T19:47:27.332 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T19:47:27.333 INFO:teuthology.orchestra.run.smithi083.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T19:47:27.393 INFO:teuthology.orchestra.run.smithi083.stdout:Dependencies resolved. 2024-04-26T19:47:27.401 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-26T19:47:27.401 INFO:teuthology.orchestra.run.smithi083.stdout: Package Arch Version Repository Size 2024-04-26T19:47:27.401 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout:Installing: 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T19:47:27.402 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T19:47:27.403 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrading: 2024-04-26T19:47:27.404 INFO:teuthology.orchestra.run.smithi083.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout:Installing dependencies: 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T19:47:27.405 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T19:47:27.406 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T19:47:27.407 INFO:teuthology.orchestra.run.smithi083.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T19:47:27.408 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T19:47:27.409 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T19:47:27.410 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T19:47:27.411 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T19:47:27.412 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout:Installing weak dependencies: 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction Summary 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout:======================================================================================= 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout:Install 148 Packages 2024-04-26T19:47:27.413 INFO:teuthology.orchestra.run.smithi083.stdout:Upgrade 4 Packages 2024-04-26T19:47:27.414 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:47:27.414 INFO:teuthology.orchestra.run.smithi083.stdout:Total download size: 222 M 2024-04-26T19:47:27.414 INFO:teuthology.orchestra.run.smithi083.stdout:Downloading Packages: 2024-04-26T19:47:28.315 INFO:teuthology.orchestra.run.smithi083.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-04-26T19:47:28.724 INFO:teuthology.orchestra.run.smithi083.stdout:(2/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 8.3 MB/s | 5.4 MB 00:00 2024-04-26T19:47:28.791 INFO:teuthology.orchestra.run.smithi083.stdout:(3/152): ceph-immutable-object-cache-19.0.0-330 2.1 MB/s | 145 kB 00:00 2024-04-26T19:47:29.074 INFO:teuthology.orchestra.run.smithi083.stdout:(4/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 8.3 MB/s | 2.4 MB 00:00 2024-04-26T19:47:29.233 INFO:teuthology.orchestra.run.smithi083.stdout:(5/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 9.6 MB/s | 1.5 MB 00:00 2024-04-26T19:47:29.425 INFO:teuthology.orchestra.run.smithi083.stdout:(6/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 802 kB/s | 889 kB 00:01 2024-04-26T19:47:29.733 INFO:teuthology.orchestra.run.smithi083.stdout:(7/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 13 MB/s | 21 MB 00:01 2024-04-26T19:47:29.809 INFO:teuthology.orchestra.run.smithi083.stdout:(8/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-04-26T19:47:29.859 INFO:teuthology.orchestra.run.smithi083.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 504 kB/s | 25 kB 00:00 2024-04-26T19:47:30.248 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 6.2 MB 00:01 2024-04-26T19:47:30.509 INFO:teuthology.orchestra.run.smithi083.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 14 MB/s | 11 MB 00:00 2024-04-26T19:47:30.568 INFO:teuthology.orchestra.run.smithi083.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 225 kB 00:00 2024-04-26T19:47:30.635 INFO:teuthology.orchestra.run.smithi083.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 407 kB/s | 27 kB 00:00 2024-04-26T19:47:30.701 INFO:teuthology.orchestra.run.smithi083.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 477 kB/s | 32 kB 00:00 2024-04-26T19:47:30.795 INFO:teuthology.orchestra.run.smithi083.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 7.8 MB/s | 747 kB 00:00 2024-04-26T19:47:30.871 INFO:teuthology.orchestra.run.smithi083.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 2.1 MB/s | 163 kB 00:00 2024-04-26T19:47:30.946 INFO:teuthology.orchestra.run.smithi083.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.7 MB/s | 127 kB 00:00 2024-04-26T19:47:31.038 INFO:teuthology.orchestra.run.smithi083.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 5.3 MB/s | 501 kB 00:00 2024-04-26T19:47:31.505 INFO:teuthology.orchestra.run.smithi083.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 12 MB/s | 5.4 MB 00:00 2024-04-26T19:47:31.564 INFO:teuthology.orchestra.run.smithi083.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 775 kB/s | 45 kB 00:00 2024-04-26T19:47:31.630 INFO:teuthology.orchestra.run.smithi083.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 2.0 MB/s | 135 kB 00:00 2024-04-26T19:47:31.706 INFO:teuthology.orchestra.run.smithi083.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T19:47:31.789 INFO:teuthology.orchestra.run.smithi083.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T19:47:31.865 INFO:teuthology.orchestra.run.smithi083.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.9 MB/s | 298 kB 00:00 2024-04-26T19:47:31.924 INFO:teuthology.orchestra.run.smithi083.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 99 kB 00:00 2024-04-26T19:47:31.983 INFO:teuthology.orchestra.run.smithi083.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T19:47:32.266 INFO:teuthology.orchestra.run.smithi083.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 11 MB/s | 3.1 MB 00:00 2024-04-26T19:47:32.333 INFO:teuthology.orchestra.run.smithi083.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.5 MB/s | 171 kB 00:00 2024-04-26T19:47:32.442 INFO:teuthology.orchestra.run.smithi083.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 316 kB/s | 34 kB 00:00 2024-04-26T19:47:32.884 INFO:teuthology.orchestra.run.smithi083.stdout:(29/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 16 MB/s | 49 MB 00:03 2024-04-26T19:47:32.909 INFO:teuthology.orchestra.run.smithi083.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 304 kB/s | 142 kB 00:00 2024-04-26T19:47:33.176 INFO:teuthology.orchestra.run.smithi083.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 12 MB/s | 3.5 MB 00:00 2024-04-26T19:47:33.251 INFO:teuthology.orchestra.run.smithi083.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 3.2 MB/s | 249 kB 00:00 2024-04-26T19:47:33.310 INFO:teuthology.orchestra.run.smithi083.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 850 kB/s | 49 kB 00:00 2024-04-26T19:47:33.419 INFO:teuthology.orchestra.run.smithi083.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-04-26T19:47:33.444 INFO:teuthology.orchestra.run.smithi083.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3306.gb 123 kB/s | 16 kB 00:00 2024-04-26T19:47:33.469 INFO:teuthology.orchestra.run.smithi083.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 5.4 MB/s | 273 kB 00:00 2024-04-26T19:47:33.511 INFO:teuthology.orchestra.run.smithi083.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 11 MB/s | 762 kB 00:00 2024-04-26T19:47:33.536 INFO:teuthology.orchestra.run.smithi083.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 272 kB/s | 18 kB 00:00 2024-04-26T19:47:33.662 INFO:teuthology.orchestra.run.smithi083.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 534 kB/s | 80 kB 00:00 2024-04-26T19:47:33.779 INFO:teuthology.orchestra.run.smithi083.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-26T19:47:33.812 INFO:teuthology.orchestra.run.smithi083.stdout:(41/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-26T19:47:33.873 INFO:teuthology.orchestra.run.smithi079.stdout:CentOS Stream 9 - Extras packages 6.5 kB/s | 16 kB 00:02 2024-04-26T19:47:33.888 INFO:teuthology.orchestra.run.smithi083.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-26T19:47:33.955 INFO:teuthology.orchestra.run.smithi083.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-26T19:47:34.013 INFO:teuthology.orchestra.run.smithi083.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-04-26T19:47:34.055 INFO:teuthology.orchestra.run.smithi083.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-26T19:47:34.081 INFO:teuthology.orchestra.run.smithi083.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-04-26T19:47:34.106 INFO:teuthology.orchestra.run.smithi083.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-26T19:47:34.373 INFO:teuthology.orchestra.run.smithi083.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-04-26T19:47:34.440 INFO:teuthology.orchestra.run.smithi083.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-26T19:47:34.473 INFO:teuthology.orchestra.run.smithi083.stdout:(50/152): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-04-26T19:47:34.524 INFO:teuthology.orchestra.run.smithi083.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-04-26T19:47:34.557 INFO:teuthology.orchestra.run.smithi083.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-26T19:47:34.599 INFO:teuthology.orchestra.run.smithi083.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-04-26T19:47:34.666 INFO:teuthology.orchestra.run.smithi083.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-04-26T19:47:34.770 INFO:teuthology.orchestra.run.smithi083.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 108 kB/s | 11 kB 00:00 2024-04-26T19:47:34.817 INFO:teuthology.orchestra.run.smithi083.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 12 MB/s | 3.0 MB 00:00 2024-04-26T19:47:34.842 INFO:teuthology.orchestra.run.smithi083.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.2 MB/s | 160 kB 00:00 2024-04-26T19:47:34.876 INFO:teuthology.orchestra.run.smithi083.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-04-26T19:47:34.942 INFO:teuthology.orchestra.run.smithi083.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-26T19:47:34.993 INFO:teuthology.orchestra.run.smithi083.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 4.9 MB/s | 582 kB 00:00 2024-04-26T19:47:35.060 INFO:teuthology.orchestra.run.smithi083.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 218 kB/s | 25 kB 00:00 2024-04-26T19:47:35.077 INFO:teuthology.orchestra.run.smithi083.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-04-26T19:47:35.127 INFO:teuthology.orchestra.run.smithi083.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-26T19:47:35.152 INFO:teuthology.orchestra.run.smithi083.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-26T19:47:35.194 INFO:teuthology.orchestra.run.smithi083.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-04-26T19:47:35.395 INFO:teuthology.orchestra.run.smithi083.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-26T19:47:35.466 INFO:teuthology.orchestra.run.smithi079.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-26T19:47:35.520 INFO:teuthology.orchestra.run.smithi083.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-26T19:47:35.621 INFO:teuthology.orchestra.run.smithi083.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-04-26T19:47:35.687 INFO:teuthology.orchestra.run.smithi083.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 4.4 MB 00:00 2024-04-26T19:47:35.704 INFO:teuthology.orchestra.run.smithi083.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.6 MB/s | 133 kB 00:00 2024-04-26T19:47:35.838 INFO:teuthology.orchestra.run.smithi083.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 7.6 MB/s | 1.0 MB 00:00 2024-04-26T19:47:35.921 INFO:teuthology.orchestra.run.smithi083.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-04-26T19:47:36.038 INFO:teuthology.orchestra.run.smithi083.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 16 MB/s | 5.8 MB 00:00 2024-04-26T19:47:36.122 INFO:teuthology.orchestra.run.smithi083.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-26T19:47:36.197 INFO:teuthology.orchestra.run.smithi083.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-04-26T19:47:36.223 INFO:teuthology.orchestra.run.smithi083.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-26T19:47:36.273 INFO:teuthology.orchestra.run.smithi083.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 340 kB/s | 26 kB 00:00 2024-04-26T19:47:36.298 INFO:teuthology.orchestra.run.smithi083.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-04-26T19:47:36.340 INFO:teuthology.orchestra.run.smithi083.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-26T19:47:36.366 INFO:teuthology.orchestra.run.smithi083.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-04-26T19:47:36.466 INFO:teuthology.orchestra.run.smithi083.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-26T19:47:36.541 INFO:teuthology.orchestra.run.smithi083.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 5.1 MB 00:00 2024-04-26T19:47:36.567 INFO:teuthology.orchestra.run.smithi083.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-26T19:47:36.600 INFO:teuthology.orchestra.run.smithi083.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T19:47:36.625 INFO:teuthology.orchestra.run.smithi083.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-04-26T19:47:36.692 INFO:teuthology.orchestra.run.smithi083.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-26T19:47:36.717 INFO:teuthology.orchestra.run.smithi083.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-26T19:47:36.768 INFO:teuthology.orchestra.run.smithi083.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-26T19:47:36.793 INFO:teuthology.orchestra.run.smithi083.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-04-26T19:47:36.843 INFO:teuthology.orchestra.run.smithi083.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 715 kB/s | 54 kB 00:00 2024-04-26T19:47:36.902 INFO:teuthology.orchestra.run.smithi083.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-04-26T19:47:36.969 INFO:teuthology.orchestra.run.smithi083.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 661 kB/s | 44 kB 00:00 2024-04-26T19:47:37.036 INFO:teuthology.orchestra.run.smithi083.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-26T19:47:37.095 INFO:teuthology.orchestra.run.smithi083.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-04-26T19:47:37.154 INFO:teuthology.orchestra.run.smithi083.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-26T19:47:37.221 INFO:teuthology.orchestra.run.smithi083.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-26T19:47:37.304 INFO:teuthology.orchestra.run.smithi083.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 694 kB/s | 58 kB 00:00 2024-04-26T19:47:37.380 INFO:teuthology.orchestra.run.smithi083.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-26T19:47:37.613 INFO:teuthology.orchestra.run.smithi083.stdout:(99/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-26T19:47:37.639 INFO:teuthology.orchestra.run.smithi083.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 246 kB/s | 64 kB 00:00 2024-04-26T19:47:37.672 INFO:teuthology.orchestra.run.smithi083.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 767 kB/s | 45 kB 00:00 2024-04-26T19:47:38.147 INFO:teuthology.orchestra.run.smithi083.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 44 kB/s | 22 kB 00:00 2024-04-26T19:47:38.315 INFO:teuthology.orchestra.run.smithi083.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 3.2 MB/s | 548 kB 00:00 2024-04-26T19:47:38.473 INFO:teuthology.orchestra.run.smithi083.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-04-26T19:47:38.582 INFO:teuthology.orchestra.run.smithi083.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-04-26T19:47:39.157 INFO:teuthology.orchestra.run.smithi083.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 582 kB/s | 863 kB 00:01 2024-04-26T19:47:39.257 INFO:teuthology.orchestra.run.smithi083.stdout:(107/152): libarrow-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-04-26T19:47:39.316 INFO:teuthology.orchestra.run.smithi083.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 164 kB/s | 26 kB 00:00 2024-04-26T19:47:39.408 INFO:teuthology.orchestra.run.smithi083.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 326 kB/s | 49 kB 00:00 2024-04-26T19:47:39.466 INFO:teuthology.orchestra.run.smithi083.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 452 kB/s | 67 kB 00:00 2024-04-26T19:47:39.583 INFO:teuthology.orchestra.run.smithi083.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 862 kB/s | 151 kB 00:00 2024-04-26T19:47:39.775 INFO:teuthology.orchestra.run.smithi083.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.8 MB/s | 548 kB 00:00 2024-04-26T19:47:39.809 INFO:teuthology.orchestra.run.smithi083.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.4 MB/s | 839 kB 00:00 2024-04-26T19:47:39.942 INFO:teuthology.orchestra.run.smithi083.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 260 kB/s | 43 kB 00:00 2024-04-26T19:47:39.967 INFO:teuthology.orchestra.run.smithi083.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 240 kB/s | 32 kB 00:00 2024-04-26T19:47:40.085 INFO:teuthology.orchestra.run.smithi083.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 106 kB/s | 15 kB 00:00 2024-04-26T19:47:40.143 INFO:teuthology.orchestra.run.smithi083.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-04-26T19:47:40.277 INFO:teuthology.orchestra.run.smithi083.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-04-26T19:47:40.335 INFO:teuthology.orchestra.run.smithi083.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.6 MB/s | 303 kB 00:00 2024-04-26T19:47:40.394 INFO:teuthology.orchestra.run.smithi083.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-26T19:47:40.494 INFO:teuthology.orchestra.run.smithi083.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.3 MB/s | 206 kB 00:00 2024-04-26T19:47:40.532 INFO:teuthology.orchestra.run.smithi079.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-04-26T19:47:40.619 INFO:teuthology.orchestra.run.smithi083.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-26T19:47:40.720 INFO:teuthology.orchestra.run.smithi083.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-26T19:47:40.820 INFO:teuthology.orchestra.run.smithi083.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-04-26T19:47:40.921 INFO:teuthology.orchestra.run.smithi083.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-26T19:47:41.021 INFO:teuthology.orchestra.run.smithi083.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 195 kB/s | 19 kB 00:00 2024-04-26T19:47:41.121 INFO:teuthology.orchestra.run.smithi083.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-26T19:47:41.205 INFO:teuthology.orchestra.run.smithi083.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.5 MB/s | 2.0 MB 00:00 2024-04-26T19:47:41.313 INFO:teuthology.orchestra.run.smithi083.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 431 kB/s | 46 kB 00:00 2024-04-26T19:47:41.397 INFO:teuthology.orchestra.run.smithi083.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 3.7 MB/s | 1.0 MB 00:00 2024-04-26T19:47:41.455 INFO:teuthology.orchestra.run.smithi083.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 556 kB/s | 79 kB 00:00 2024-04-26T19:47:41.522 INFO:teuthology.orchestra.run.smithi083.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 461 kB/s | 58 kB 00:00 2024-04-26T19:47:41.723 INFO:teuthology.orchestra.run.smithi083.stdout:(133/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x 1.4 MB/s | 17 MB 00:12 2024-04-26T19:47:41.790 INFO:teuthology.orchestra.run.smithi083.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 818 kB/s | 272 kB 00:00 2024-04-26T19:47:41.815 INFO:teuthology.orchestra.run.smithi083.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 56 kB/s | 16 kB 00:00 2024-04-26T19:47:41.898 INFO:teuthology.orchestra.run.smithi083.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 516 kB/s | 90 kB 00:00 2024-04-26T19:47:41.924 INFO:teuthology.orchestra.run.smithi083.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 214 kB/s | 28 kB 00:00 2024-04-26T19:47:41.974 INFO:teuthology.orchestra.run.smithi083.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 228 kB/s | 31 kB 00:00 2024-04-26T19:47:42.091 INFO:teuthology.orchestra.run.smithi083.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 986 kB/s | 188 kB 00:00 2024-04-26T19:47:42.116 INFO:teuthology.orchestra.run.smithi083.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 352 kB/s | 59 kB 00:00 2024-04-26T19:47:42.141 INFO:teuthology.orchestra.run.smithi083.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 251 kB/s | 36 kB 00:00 2024-04-26T19:47:42.292 INFO:teuthology.orchestra.run.smithi083.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 255 kB/s | 51 kB 00:00 2024-04-26T19:47:42.325 INFO:teuthology.orchestra.run.smithi083.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 560 kB/s | 90 kB 00:00 2024-04-26T19:47:42.376 INFO:teuthology.orchestra.run.smithi083.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.1 MB/s | 230 kB 00:00 2024-04-26T19:47:42.476 INFO:teuthology.orchestra.run.smithi083.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 158 kB/s | 20 kB 00:00 2024-04-26T19:47:42.543 INFO:teuthology.orchestra.run.smithi083.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.7 MB/s | 427 kB 00:00 2024-04-26T19:47:42.568 INFO:teuthology.orchestra.run.smithi083.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-26T19:47:42.674 INFO:teuthology.orchestra.run.smithi079.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:47:42.675 INFO:teuthology.orchestra.run.smithi079.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T19:47:42.686 INFO:teuthology.orchestra.run.smithi079.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T19:47:42.686 INFO:teuthology.orchestra.run.smithi079.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T19:47:42.687 INFO:teuthology.orchestra.run.smithi079.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T19:47:42.688 INFO:teuthology.orchestra.run.smithi079.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T19:47:42.690 INFO:teuthology.orchestra.run.smithi079.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T19:47:42.692 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T19:47:42.693 INFO:teuthology.orchestra.run.smithi079.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T19:47:42.753 INFO:teuthology.orchestra.run.smithi079.stdout:Dependencies resolved. 2024-04-26T19:47:42.762 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-04-26T19:47:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: Package Arch Version Repository Size 2024-04-26T19:47:42.762 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-04-26T19:47:42.762 INFO:teuthology.orchestra.run.smithi079.stdout:Installing: 2024-04-26T19:47:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T19:47:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: ceph x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 6.6 k 2024-04-26T19:47:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:47:42.762 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 889 k 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 145 k 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 1.5 M 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 142 k 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 3.5 M 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 7.4 M 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 49 k 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 11 M 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 49 M 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 273 k 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 762 k 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 225 k 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 27 k 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 18 k 2024-04-26T19:47:42.763 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 32 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 747 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 127 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 161 k 2024-04-26T19:47:42.764 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 321 k 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 298 k 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 99 k 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 85 k 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 171 k 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrading: 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: librados2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.4 M 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 3.1 M 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout:Installing dependencies: 2024-04-26T19:47:42.765 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 21 M 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 34 k 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 2.4 M 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 249 k 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 4.7 M 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 17 M 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3306.gbc612ffb.el9 ceph-noarch 16 k 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 25 k 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T19:47:42.766 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 163 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 501 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T19:47:42.767 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2 x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 5.4 M 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T19:47:42.768 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 45 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common x86_64 2:19.0.0-3306.gbc612ffb.el9 ceph 135 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T19:47:42.769 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T19:47:42.770 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T19:47:42.771 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T19:47:42.772 INFO:teuthology.orchestra.run.smithi079.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout:Installing weak dependencies: 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction Summary 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout:======================================================================================= 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout:Install 148 Packages 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout:Upgrade 4 Packages 2024-04-26T19:47:42.773 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:47:42.774 INFO:teuthology.orchestra.run.smithi079.stdout:Total download size: 222 M 2024-04-26T19:47:42.774 INFO:teuthology.orchestra.run.smithi079.stdout:Downloading Packages: 2024-04-26T19:47:42.777 INFO:teuthology.orchestra.run.smithi083.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-04-26T19:47:42.852 INFO:teuthology.orchestra.run.smithi083.stdout:(149/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 11 MB/s | 3.1 MB 00:00 2024-04-26T19:47:43.036 INFO:teuthology.orchestra.run.smithi083.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 1.3 MB 00:00 2024-04-26T19:47:43.111 INFO:teuthology.orchestra.run.smithi083.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 2.4 MB 00:00 2024-04-26T19:47:43.736 INFO:teuthology.orchestra.run.smithi083.stdout:(152/152): librados2-19.0.0-3306.gbc612ffb.el9. 2.8 MB/s | 3.4 MB 00:01 2024-04-26T19:47:43.739 INFO:teuthology.orchestra.run.smithi083.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:47:43.739 INFO:teuthology.orchestra.run.smithi083.stdout:Total 14 MB/s | 222 MB 00:16 2024-04-26T19:47:44.050 INFO:teuthology.orchestra.run.smithi079.stdout:(1/152): ceph-19.0.0-3306.gbc612ffb.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-26T19:47:44.542 INFO:teuthology.orchestra.run.smithi079.stdout:(2/152): ceph-fuse-19.0.0-3306.gbc612ffb.el9.x8 1.8 MB/s | 889 kB 00:00 2024-04-26T19:47:44.626 INFO:teuthology.orchestra.run.smithi079.stdout:(3/152): ceph-base-19.0.0-3306.gbc612ffb.el9.x8 5.0 MB/s | 5.4 MB 00:01 2024-04-26T19:47:44.651 INFO:teuthology.orchestra.run.smithi079.stdout:(4/152): ceph-immutable-object-cache-19.0.0-330 1.3 MB/s | 145 kB 00:00 2024-04-26T19:47:44.837 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction check 2024-04-26T19:47:44.876 INFO:teuthology.orchestra.run.smithi079.stdout:(5/152): ceph-mds-19.0.0-3306.gbc612ffb.el9.x86 9.4 MB/s | 2.4 MB 00:00 2024-04-26T19:47:44.979 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction check succeeded. 2024-04-26T19:47:44.979 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction test 2024-04-26T19:47:45.335 INFO:teuthology.orchestra.run.smithi079.stdout:(6/152): ceph-mgr-19.0.0-3306.gbc612ffb.el9.x86 2.2 MB/s | 1.5 MB 00:00 2024-04-26T19:47:45.401 INFO:teuthology.orchestra.run.smithi079.stdout:(7/152): ceph-mon-19.0.0-3306.gbc612ffb.el9.x86 9.0 MB/s | 4.7 MB 00:00 2024-04-26T19:47:45.902 INFO:teuthology.orchestra.run.smithi079.stdout:(8/152): ceph-common-19.0.0-3306.gbc612ffb.el9. 9.0 MB/s | 21 MB 00:02 2024-04-26T19:47:45.952 INFO:teuthology.orchestra.run.smithi079.stdout:(9/152): ceph-selinux-19.0.0-3306.gbc612ffb.el9 506 kB/s | 25 kB 00:00 2024-04-26T19:47:46.302 INFO:teuthology.orchestra.run.smithi079.stdout:(10/152): ceph-radosgw-19.0.0-3306.gbc612ffb.el 12 MB/s | 11 MB 00:00 2024-04-26T19:47:46.402 INFO:teuthology.orchestra.run.smithi079.stdout:(11/152): cephfs-mirror-19.0.0-3306.gbc612ffb.e 2.2 MB/s | 225 kB 00:00 2024-04-26T19:47:46.486 INFO:teuthology.orchestra.run.smithi079.stdout:(12/152): cephfs-shell-19.0.0-3306.gbc612ffb.el 325 kB/s | 27 kB 00:00 2024-04-26T19:47:46.561 INFO:teuthology.orchestra.run.smithi079.stdout:(13/152): libcephfs-devel-19.0.0-3306.gbc612ffb 423 kB/s | 32 kB 00:00 2024-04-26T19:47:46.661 INFO:teuthology.orchestra.run.smithi079.stdout:(14/152): libcephfs2-19.0.0-3306.gbc612ffb.el9. 7.3 MB/s | 747 kB 00:00 2024-04-26T19:47:46.741 INFO:teuthology.orchestra.run.smithi083.stdout:Transaction test succeeded. 2024-04-26T19:47:46.741 INFO:teuthology.orchestra.run.smithi083.stdout:Running transaction 2024-04-26T19:47:46.745 INFO:teuthology.orchestra.run.smithi079.stdout:(15/152): libcephsqlite-19.0.0-3306.gbc612ffb.e 1.9 MB/s | 163 kB 00:00 2024-04-26T19:47:46.829 INFO:teuthology.orchestra.run.smithi079.stdout:(16/152): librados-devel-19.0.0-3306.gbc612ffb. 1.5 MB/s | 127 kB 00:00 2024-04-26T19:47:46.929 INFO:teuthology.orchestra.run.smithi079.stdout:(17/152): libradosstriper1-19.0.0-3306.gbc612ff 4.9 MB/s | 501 kB 00:00 2024-04-26T19:47:47.471 INFO:teuthology.orchestra.run.smithi079.stdout:(18/152): librgw2-19.0.0-3306.gbc612ffb.el9.x86 9.9 MB/s | 5.4 MB 00:00 2024-04-26T19:47:47.563 INFO:teuthology.orchestra.run.smithi079.stdout:(19/152): python3-ceph-argparse-19.0.0-3306.gbc 493 kB/s | 45 kB 00:00 2024-04-26T19:47:47.647 INFO:teuthology.orchestra.run.smithi079.stdout:(20/152): python3-ceph-common-19.0.0-3306.gbc61 1.6 MB/s | 135 kB 00:00 2024-04-26T19:47:47.722 INFO:teuthology.orchestra.run.smithi079.stdout:(21/152): python3-cephfs-19.0.0-3306.gbc612ffb. 2.1 MB/s | 161 kB 00:00 2024-04-26T19:47:47.805 INFO:teuthology.orchestra.run.smithi079.stdout:(22/152): python3-rados-19.0.0-3306.gbc612ffb.e 3.8 MB/s | 321 kB 00:00 2024-04-26T19:47:47.889 INFO:teuthology.orchestra.run.smithi079.stdout:(23/152): python3-rbd-19.0.0-3306.gbc612ffb.el9 3.5 MB/s | 298 kB 00:00 2024-04-26T19:47:47.973 INFO:teuthology.orchestra.run.smithi079.stdout:(24/152): python3-rgw-19.0.0-3306.gbc612ffb.el9 1.2 MB/s | 99 kB 00:00 2024-04-26T19:47:48.064 INFO:teuthology.orchestra.run.smithi079.stdout:(25/152): rbd-fuse-19.0.0-3306.gbc612ffb.el9.x8 932 kB/s | 85 kB 00:00 2024-04-26T19:47:48.406 INFO:teuthology.orchestra.run.smithi079.stdout:(26/152): rbd-mirror-19.0.0-3306.gbc612ffb.el9. 9.0 MB/s | 3.1 MB 00:00 2024-04-26T19:47:48.490 INFO:teuthology.orchestra.run.smithi079.stdout:(27/152): rbd-nbd-19.0.0-3306.gbc612ffb.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-26T19:47:48.565 INFO:teuthology.orchestra.run.smithi079.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3306.g 456 kB/s | 34 kB 00:00 2024-04-26T19:47:48.640 INFO:teuthology.orchestra.run.smithi079.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3306.gbc612ff 1.9 MB/s | 142 kB 00:00 2024-04-26T19:47:49.074 INFO:teuthology.orchestra.run.smithi079.stdout:(30/152): ceph-test-19.0.0-3306.gbc612ffb.el9.x 16 MB/s | 49 MB 00:03 2024-04-26T19:47:49.149 INFO:teuthology.orchestra.run.smithi079.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3306.gbc612 7.0 MB/s | 3.5 MB 00:00 2024-04-26T19:47:49.258 INFO:teuthology.orchestra.run.smithi079.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3306.gbc 2.2 MB/s | 249 kB 00:00 2024-04-26T19:47:49.350 INFO:teuthology.orchestra.run.smithi079.stdout:(33/152): ceph-mgr-rook-19.0.0-3306.gbc612ffb.e 540 kB/s | 49 kB 00:00 2024-04-26T19:47:49.425 INFO:teuthology.orchestra.run.smithi079.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3306.gb 220 kB/s | 16 kB 00:00 2024-04-26T19:47:49.559 INFO:teuthology.orchestra.run.smithi079.stdout:(35/152): ceph-mgr-diskprediction-local-19.0.0- 15 MB/s | 7.4 MB 00:00 2024-04-26T19:47:49.584 INFO:teuthology.orchestra.run.smithi079.stdout:(36/152): ceph-volume-19.0.0-3306.gbc612ffb.el9 1.7 MB/s | 273 kB 00:00 2024-04-26T19:47:49.642 INFO:teuthology.orchestra.run.smithi079.stdout:(37/152): cephadm-19.0.0-3306.gbc612ffb.el9.noa 8.9 MB/s | 762 kB 00:00 2024-04-26T19:47:49.668 INFO:teuthology.orchestra.run.smithi079.stdout:(38/152): cephfs-top-19.0.0-3306.gbc612ffb.el9. 217 kB/s | 18 kB 00:00 2024-04-26T19:47:49.877 INFO:teuthology.orchestra.run.smithi079.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 343 kB/s | 80 kB 00:00 2024-04-26T19:47:49.969 INFO:teuthology.orchestra.run.smithi079.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-04-26T19:47:50.002 INFO:teuthology.orchestra.run.smithi079.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-26T19:47:50.161 INFO:teuthology.orchestra.run.smithi079.stdout:(42/152): ceph-osd-19.0.0-3306.gbc612ffb.el9.x8 3.5 MB/s | 17 MB 00:04 2024-04-26T19:47:50.220 INFO:teuthology.orchestra.run.smithi079.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-04-26T19:47:50.245 INFO:teuthology.orchestra.run.smithi079.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-26T19:47:50.329 INFO:teuthology.orchestra.run.smithi079.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-26T19:47:50.354 INFO:teuthology.orchestra.run.smithi079.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-26T19:47:50.388 INFO:teuthology.orchestra.run.smithi079.stdout:(47/152): python3-cryptography-36.0.1-4.el9.x86 3.2 MB/s | 1.2 MB 00:00 2024-04-26T19:47:50.438 INFO:teuthology.orchestra.run.smithi079.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-26T19:47:51.088 INFO:teuthology.orchestra.run.smithi079.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 164 kB/s | 107 kB 00:00 2024-04-26T19:47:51.605 INFO:teuthology.orchestra.run.smithi079.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 42 kB/s | 21 kB 00:00 2024-04-26T19:47:51.638 INFO:teuthology.orchestra.run.smithi079.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 537 kB/s | 689 kB 00:01 2024-04-26T19:47:51.939 INFO:teuthology.orchestra.run.smithi079.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 608 kB/s | 943 kB 00:01 2024-04-26T19:47:52.189 INFO:teuthology.orchestra.run.smithi079.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-04-26T19:47:52.514 INFO:teuthology.orchestra.run.smithi079.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 345 kB/s | 313 kB 00:00 2024-04-26T19:47:52.624 INFO:teuthology.orchestra.run.smithi083.stdout: Preparing : 1/1 2024-04-26T19:47:52.698 INFO:teuthology.orchestra.run.smithi079.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-04-26T19:47:52.715 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T19:47:52.753 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T19:47:52.806 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T19:47:52.849 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T19:47:52.875 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T19:47:52.917 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T19:47:52.998 INFO:teuthology.orchestra.run.smithi079.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-04-26T19:47:53.235 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T19:47:53.343 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:47:53.463 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:47:53.498 INFO:teuthology.orchestra.run.smithi079.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 200 kB/s | 160 kB 00:00 2024-04-26T19:47:53.502 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:47:53.593 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:47:53.598 INFO:teuthology.orchestra.run.smithi079.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 75 kB/s | 45 kB 00:00 2024-04-26T19:47:53.623 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:47:53.769 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T19:47:53.812 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T19:47:53.840 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:47:53.934 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:47:53.957 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:47:54.003 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:47:54.025 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:47:54.048 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:47:54.242 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:47:54.275 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:47:54.382 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T19:47:54.486 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T19:47:54.516 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T19:47:54.550 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T19:47:54.627 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T19:47:54.667 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T19:47:54.710 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T19:47:54.743 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T19:47:54.771 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T19:47:54.841 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T19:47:54.894 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T19:47:54.924 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T19:47:54.968 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T19:47:55.031 INFO:teuthology.orchestra.run.smithi079.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 432 kB/s | 662 kB 00:01 2024-04-26T19:47:55.045 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T19:47:55.065 INFO:teuthology.orchestra.run.smithi079.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 397 kB/s | 582 kB 00:01 2024-04-26T19:47:55.079 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T19:47:55.122 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T19:47:55.151 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T19:47:55.260 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T19:47:55.327 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T19:47:55.444 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T19:47:55.548 INFO:teuthology.orchestra.run.smithi079.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 49 kB/s | 25 kB 00:00 2024-04-26T19:47:55.607 INFO:teuthology.orchestra.run.smithi079.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 88 kB/s | 47 kB 00:00 2024-04-26T19:47:55.675 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T19:47:55.753 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T19:47:56.032 INFO:teuthology.orchestra.run.smithi079.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 748 kB/s | 3.0 MB 00:04 2024-04-26T19:47:56.315 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T19:47:56.397 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T19:47:56.426 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T19:47:56.474 INFO:teuthology.orchestra.run.smithi079.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 217 kB/s | 188 kB 00:00 2024-04-26T19:47:56.477 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T19:47:56.515 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T19:47:56.516 INFO:teuthology.orchestra.run.smithi079.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 302 kB/s | 292 kB 00:00 2024-04-26T19:47:56.557 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T19:47:56.616 INFO:teuthology.orchestra.run.smithi079.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 60 kB/s | 35 kB 00:00 2024-04-26T19:47:56.738 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T19:47:56.793 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T19:47:56.847 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T19:47:56.929 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T19:47:56.961 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T19:47:56.997 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T19:47:57.044 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T19:47:57.080 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T19:47:57.123 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T19:47:57.173 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T19:47:57.309 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T19:47:57.354 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T19:47:57.375 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T19:47:57.417 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T19:47:57.519 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T19:47:57.559 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T19:47:57.599 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T19:47:57.625 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T19:47:57.650 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T19:47:57.724 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T19:47:57.851 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T19:47:57.891 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T19:47:57.910 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T19:47:57.944 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T19:47:58.067 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T19:47:58.233 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T19:47:58.262 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T19:47:58.289 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T19:47:58.338 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T19:47:58.380 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T19:47:58.415 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T19:47:58.504 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T19:47:58.553 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T19:47:58.585 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T19:47:58.632 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T19:47:58.774 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T19:47:58.819 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T19:47:59.416 INFO:teuthology.orchestra.run.smithi079.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 581 kB/s | 1.6 MB 00:02 2024-04-26T19:47:59.541 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T19:47:59.592 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T19:47:59.625 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T19:47:59.722 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T19:47:59.775 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T19:47:59.862 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T19:47:59.895 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T19:48:00.073 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T19:48:00.100 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T19:48:00.178 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T19:48:00.830 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T19:48:00.916 INFO:teuthology.orchestra.run.smithi079.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 608 kB/s | 911 kB 00:01 2024-04-26T19:48:00.942 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T19:48:01.600 INFO:teuthology.orchestra.run.smithi079.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 195 kB/s | 133 kB 00:00 2024-04-26T19:48:02.091 INFO:teuthology.orchestra.run.smithi079.stdout:(70/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 813 kB/s | 4.4 MB 00:05 2024-04-26T19:48:02.166 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T19:48:02.285 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T19:48:02.316 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T19:48:02.350 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T19:48:02.533 INFO:teuthology.orchestra.run.smithi079.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 801 kB/s | 4.7 MB 00:06 2024-04-26T19:48:02.662 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T19:48:02.713 INFO:teuthology.orchestra.run.smithi083.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:48:02.786 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:48:02.809 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T19:48:02.845 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T19:48:03.142 INFO:teuthology.orchestra.run.smithi079.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 144 kB/s | 87 kB 00:00 2024-04-26T19:48:03.325 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T19:48:03.357 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:48:03.429 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:48:03.459 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T19:48:03.758 INFO:teuthology.orchestra.run.smithi079.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 620 kB/s | 1.0 MB 00:01 2024-04-26T19:48:04.525 INFO:teuthology.orchestra.run.smithi079.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 318 kB/s | 244 kB 00:00 2024-04-26T19:48:05.334 INFO:teuthology.orchestra.run.smithi079.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 307 kB/s | 248 kB 00:00 2024-04-26T19:48:05.767 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:48:05.814 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:48:05.843 INFO:teuthology.orchestra.run.smithi079.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 50 kB/s | 26 kB 00:00 2024-04-26T19:48:06.326 INFO:teuthology.orchestra.run.smithi079.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 40 kB/s | 19 kB 00:00 2024-04-26T19:48:06.363 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:48:06.401 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:48:06.491 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:48:06.491 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T19:48:06.491 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T19:48:06.491 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:06.519 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:48:07.060 INFO:teuthology.orchestra.run.smithi079.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 235 kB/s | 172 kB 00:00 2024-04-26T19:48:07.593 INFO:teuthology.orchestra.run.smithi079.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 65 kB/s | 35 kB 00:00 2024-04-26T19:48:08.460 INFO:teuthology.orchestra.run.smithi079.stdout:(80/152): postgresql-server-13.14-1.el9.x86_64. 863 kB/s | 5.8 MB 00:06 2024-04-26T19:48:09.302 INFO:teuthology.orchestra.run.smithi079.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 298 kB/s | 250 kB 00:00 2024-04-26T19:48:10.060 INFO:teuthology.orchestra.run.smithi079.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 293 kB/s | 222 kB 00:00 2024-04-26T19:48:10.594 INFO:teuthology.orchestra.run.smithi079.stdout:(83/152): python3-babel-2.9.1-2.el9.noarch.rpm 819 kB/s | 6.0 MB 00:07 2024-04-26T19:48:10.686 INFO:teuthology.orchestra.run.smithi079.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 123 kB/s | 77 kB 00:00 2024-04-26T19:48:11.177 INFO:teuthology.orchestra.run.smithi079.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 72 kB/s | 42 kB 00:00 2024-04-26T19:48:11.569 INFO:teuthology.orchestra.run.smithi079.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 305 kB/s | 269 kB 00:00 2024-04-26T19:48:11.911 INFO:teuthology.orchestra.run.smithi079.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-04-26T19:48:12.412 INFO:teuthology.orchestra.run.smithi079.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 331 kB/s | 279 kB 00:00 2024-04-26T19:48:12.462 INFO:teuthology.orchestra.run.smithi079.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 93 kB/s | 51 kB 00:00 2024-04-26T19:48:12.995 INFO:teuthology.orchestra.run.smithi079.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 92 kB/s | 54 kB 00:00 2024-04-26T19:48:13.537 INFO:teuthology.orchestra.run.smithi079.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 77 kB/s | 42 kB 00:00 2024-04-26T19:48:13.762 INFO:teuthology.orchestra.run.smithi079.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 846 kB/s | 5.1 MB 00:06 2024-04-26T19:48:14.121 INFO:teuthology.orchestra.run.smithi079.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 76 kB/s | 44 kB 00:00 2024-04-26T19:48:14.538 INFO:teuthology.orchestra.run.smithi079.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 295 kB/s | 221 kB 00:00 2024-04-26T19:48:14.663 INFO:teuthology.orchestra.run.smithi079.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 66 kB/s | 36 kB 00:00 2024-04-26T19:48:15.113 INFO:teuthology.orchestra.run.smithi079.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-04-26T19:48:15.522 INFO:teuthology.orchestra.run.smithi079.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 355 kB/s | 305 kB 00:00 2024-04-26T19:48:15.680 INFO:teuthology.orchestra.run.smithi079.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 102 kB/s | 58 kB 00:00 2024-04-26T19:48:16.164 INFO:teuthology.orchestra.run.smithi079.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 146 kB/s | 94 kB 00:00 2024-04-26T19:48:16.264 INFO:teuthology.orchestra.run.smithi079.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 109 kB/s | 64 kB 00:00 2024-04-26T19:48:16.356 INFO:teuthology.orchestra.run.smithi079.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-04-26T19:48:16.615 INFO:teuthology.orchestra.run.smithi079.stdout:(102/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.6 MB/s | 863 kB 00:00 2024-04-26T19:48:16.715 INFO:teuthology.orchestra.run.smithi079.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 81 kB/s | 45 kB 00:00 2024-04-26T19:48:16.882 INFO:teuthology.orchestra.run.smithi079.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.8 MB/s | 309 kB 00:00 2024-04-26T19:48:16.907 INFO:teuthology.orchestra.run.smithi079.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-26T19:48:16.932 INFO:teuthology.orchestra.run.smithi079.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 391 kB/s | 19 kB 00:00 2024-04-26T19:48:17.050 INFO:teuthology.orchestra.run.smithi079.stdout:(107/152): libarrow-9.0.0-11.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-04-26T19:48:17.075 INFO:teuthology.orchestra.run.smithi079.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 184 kB/s | 26 kB 00:00 2024-04-26T19:48:17.125 INFO:teuthology.orchestra.run.smithi079.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 979 kB/s | 49 kB 00:00 2024-04-26T19:48:17.150 INFO:teuthology.orchestra.run.smithi079.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 900 kB/s | 67 kB 00:00 2024-04-26T19:48:17.175 INFO:teuthology.orchestra.run.smithi079.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-04-26T19:48:17.217 INFO:teuthology.orchestra.run.smithi079.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-04-26T19:48:17.243 INFO:teuthology.orchestra.run.smithi079.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-04-26T19:48:17.268 INFO:teuthology.orchestra.run.smithi079.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 884 kB/s | 43 kB 00:00 2024-04-26T19:48:17.293 INFO:teuthology.orchestra.run.smithi079.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-04-26T19:48:17.318 INFO:teuthology.orchestra.run.smithi079.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 301 kB/s | 15 kB 00:00 2024-04-26T19:48:17.344 INFO:teuthology.orchestra.run.smithi079.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-04-26T19:48:17.369 INFO:teuthology.orchestra.run.smithi079.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-26T19:48:17.394 INFO:teuthology.orchestra.run.smithi079.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 6.0 MB/s | 303 kB 00:00 2024-04-26T19:48:17.419 INFO:teuthology.orchestra.run.smithi079.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 713 kB/s | 35 kB 00:00 2024-04-26T19:48:17.445 INFO:teuthology.orchestra.run.smithi079.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-26T19:48:17.512 INFO:teuthology.orchestra.run.smithi079.stdout:(122/152): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-04-26T19:48:17.537 INFO:teuthology.orchestra.run.smithi079.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-04-26T19:48:17.562 INFO:teuthology.orchestra.run.smithi079.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-04-26T19:48:17.587 INFO:teuthology.orchestra.run.smithi079.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-04-26T19:48:17.613 INFO:teuthology.orchestra.run.smithi079.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-04-26T19:48:17.638 INFO:teuthology.orchestra.run.smithi079.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 389 kB/s | 19 kB 00:00 2024-04-26T19:48:17.663 INFO:teuthology.orchestra.run.smithi079.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-04-26T19:48:17.705 INFO:teuthology.orchestra.run.smithi079.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-04-26T19:48:17.731 INFO:teuthology.orchestra.run.smithi079.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 701 kB/s | 46 kB 00:00 2024-04-26T19:48:17.735 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:48:17.735 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /sys 2024-04-26T19:48:17.735 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /proc 2024-04-26T19:48:17.735 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /mnt 2024-04-26T19:48:17.735 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /var/tmp 2024-04-26T19:48:17.735 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /home 2024-04-26T19:48:17.735 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /root 2024-04-26T19:48:17.735 INFO:teuthology.orchestra.run.smithi083.stdout:skipping the directory /tmp 2024-04-26T19:48:17.735 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:17.756 INFO:teuthology.orchestra.run.smithi079.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-26T19:48:17.781 INFO:teuthology.orchestra.run.smithi079.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-26T19:48:17.815 INFO:teuthology.orchestra.run.smithi079.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-04-26T19:48:17.833 INFO:teuthology.orchestra.run.smithi079.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-04-26T19:48:17.866 INFO:teuthology.orchestra.run.smithi079.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-26T19:48:17.883 INFO:teuthology.orchestra.run.smithi079.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-26T19:48:17.908 INFO:teuthology.orchestra.run.smithi079.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 739 kB/s | 31 kB 00:00 2024-04-26T19:48:17.908 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:48:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:48:17.926 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:48:17.926 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T19:48:17.926 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:17.933 INFO:teuthology.orchestra.run.smithi079.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2024-04-26T19:48:17.950 INFO:teuthology.orchestra.run.smithi079.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-26T19:48:17.975 INFO:teuthology.orchestra.run.smithi079.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 857 kB/s | 36 kB 00:00 2024-04-26T19:48:17.992 INFO:teuthology.orchestra.run.smithi079.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.2 MB/s | 51 kB 00:00 2024-04-26T19:48:18.026 INFO:teuthology.orchestra.run.smithi079.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-26T19:48:18.051 INFO:teuthology.orchestra.run.smithi079.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-26T19:48:18.093 INFO:teuthology.orchestra.run.smithi079.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.3 MB/s | 427 kB 00:00 2024-04-26T19:48:18.118 INFO:teuthology.orchestra.run.smithi079.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-26T19:48:18.144 INFO:teuthology.orchestra.run.smithi079.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-26T19:48:18.194 INFO:teuthology.orchestra.run.smithi079.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-04-26T19:48:18.703 INFO:teuthology.orchestra.run.smithi079.stdout:(148/152): librbd1-19.0.0-3306.gbc612ffb.el9.x8 6.2 MB/s | 3.1 MB 00:00 2024-04-26T19:48:18.819 INFO:teuthology.orchestra.run.smithi079.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-26T19:48:18.978 INFO:teuthology.orchestra.run.smithi079.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-04-26T19:48:19.053 INFO:teuthology.orchestra.run.smithi079.stdout:(151/152): librados2-19.0.0-3306.gbc612ffb.el9. 3.7 MB/s | 3.4 MB 00:00 2024-04-26T19:48:19.420 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:48:19.445 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:48:19.446 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:48:19.446 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T19:48:19.446 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:48:19.446 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:48:19.446 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:19.671 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:48:19.693 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:48:19.693 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:48:19.693 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T19:48:19.693 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:48:19.693 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:48:19.693 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:20.130 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:48:20.152 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:48:20.152 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:48:20.152 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T19:48:20.152 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:48:20.152 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:48:20.152 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:20.259 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T19:48:20.309 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T19:48:20.363 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T19:48:20.549 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T19:48:20.597 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T19:48:20.675 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T19:48:20.764 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:48:20.905 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:48:20.947 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T19:48:20.979 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T19:48:21.007 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T19:48:21.401 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:48:21.422 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:48:21.454 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T19:48:21.520 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T19:48:21.546 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:48:22.579 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:48:22.623 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:48:23.589 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:48:23.623 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:48:23.724 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:48:23.886 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T19:48:23.917 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:48:23.940 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:48:23.940 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:48:23.940 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T19:48:23.941 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:48:23.941 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:48:23.941 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:23.984 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:48:23.995 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:48:24.093 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T19:48:24.141 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T19:48:25.068 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T19:48:25.095 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:48:25.118 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:48:25.118 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:48:25.118 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T19:48:25.118 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:48:25.118 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:48:25.118 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:25.154 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:48:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:48:25.177 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:48:25.177 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T19:48:25.177 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:25.222 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:48:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:48:25.244 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:48:25.244 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T19:48:25.244 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:48:25.244 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:48:25.244 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:25.555 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:48:25.578 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:48:25.578 INFO:teuthology.orchestra.run.smithi083.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:48:25.578 INFO:teuthology.orchestra.run.smithi083.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T19:48:25.578 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:48:25.578 INFO:teuthology.orchestra.run.smithi083.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:48:25.578 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:30.143 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T19:48:30.184 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T19:48:30.225 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T19:48:30.257 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T19:48:30.284 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T19:48:30.463 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T19:48:30.506 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T19:48:30.536 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T19:48:30.968 INFO:teuthology.orchestra.run.smithi079.stdout:(152/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 899 kB/s | 16 MB 00:18 2024-04-26T19:48:30.973 INFO:teuthology.orchestra.run.smithi079.stdout:-------------------------------------------------------------------------------- 2024-04-26T19:48:30.973 INFO:teuthology.orchestra.run.smithi079.stdout:Total 4.6 MB/s | 222 MB 00:48 2024-04-26T19:48:31.116 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:48:31.159 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:48:31.178 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:48:32.004 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T19:48:32.050 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction check 2024-04-26T19:48:32.089 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T19:48:32.130 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T19:48:32.192 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction check succeeded. 2024-04-26T19:48:32.192 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction test 2024-04-26T19:48:32.212 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T19:48:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T19:48:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:48:32.311 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:48:32.335 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:48:32.336 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:48:32.418 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:48:32.483 INFO:teuthology.orchestra.run.smithi083.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:48:34.022 INFO:teuthology.orchestra.run.smithi079.stdout:Transaction test succeeded. 2024-04-26T19:48:34.023 INFO:teuthology.orchestra.run.smithi079.stdout:Running transaction 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:48:38.617 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T19:48:38.618 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T19:48:38.619 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T19:48:38.620 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T19:48:38.621 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T19:48:38.622 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T19:48:38.623 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T19:48:38.624 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T19:48:38.625 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T19:48:38.626 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T19:48:38.627 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T19:48:38.628 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T19:48:38.629 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:48:38.629 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T19:48:40.039 INFO:teuthology.orchestra.run.smithi079.stdout: Preparing : 1/1 2024-04-26T19:48:40.120 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T19:48:40.158 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T19:48:40.221 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T19:48:40.254 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T19:48:40.280 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T19:48:40.323 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T19:48:40.643 INFO:teuthology.orchestra.run.smithi083.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout:Upgraded: 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout:Installed: 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.644 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.645 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T19:48:40.646 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T19:48:40.647 INFO:teuthology.orchestra.run.smithi083.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T19:48:40.648 INFO:teuthology.orchestra.run.smithi083.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.649 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T19:48:40.650 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T19:48:40.651 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.652 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T19:48:40.653 INFO:teuthology.orchestra.run.smithi083.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T19:48:40.654 INFO:teuthology.orchestra.run.smithi083.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T19:48:40.655 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T19:48:40.655 INFO:teuthology.orchestra.run.smithi083.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T19:48:40.655 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:48:40.655 INFO:teuthology.orchestra.run.smithi083.stdout:Complete! 2024-04-26T19:48:40.756 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:48:40.878 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:48:40.907 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:48:40.998 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:48:41.039 DEBUG:teuthology.parallel:result is None 2024-04-26T19:48:41.039 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:48:41.185 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T19:48:41.233 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T19:48:41.261 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:48:41.363 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:48:41.387 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:48:41.449 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 14/156 2024-04-26T19:48:41.471 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:48:41.493 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T19:48:41.688 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:48:41.720 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T19:48:41.829 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T19:48:41.914 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T19:48:41.953 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T19:48:41.988 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T19:48:42.052 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T19:48:42.086 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T19:48:42.130 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T19:48:42.163 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T19:48:42.191 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T19:48:42.258 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T19:48:42.313 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T19:48:42.343 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T19:48:42.378 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T19:48:42.454 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T19:48:42.489 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 31/156 2024-04-26T19:48:42.532 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 32/156 2024-04-26T19:48:42.561 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 33/156 2024-04-26T19:48:42.630 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T19:48:42.676 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T19:48:42.749 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T19:48:42.890 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T19:48:42.945 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T19:48:43.469 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T19:48:43.557 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T19:48:43.586 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T19:48:43.629 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T19:48:43.667 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 43/156 2024-04-26T19:48:43.708 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T19:48:43.855 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T19:48:43.936 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T19:48:43.991 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T19:48:44.063 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T19:48:44.095 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T19:48:44.131 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T19:48:44.179 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T19:48:44.214 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T19:48:44.257 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T19:48:44.307 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T19:48:44.442 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T19:48:44.488 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T19:48:44.518 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T19:48:44.560 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T19:48:44.654 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T19:48:44.685 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T19:48:44.716 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T19:48:44.742 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T19:48:44.767 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T19:48:44.842 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T19:48:44.967 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T19:48:45.007 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T19:48:45.035 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T19:48:45.069 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T19:48:45.117 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T19:48:45.148 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T19:48:45.177 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T19:48:45.212 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T19:48:45.245 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T19:48:45.287 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T19:48:45.322 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T19:48:45.407 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T19:48:45.460 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T19:48:45.491 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T19:48:45.538 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T19:48:45.637 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T19:48:45.684 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T19:48:46.401 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T19:48:46.448 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T19:48:46.482 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T19:48:46.580 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T19:48:46.632 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T19:48:46.759 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T19:48:46.983 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T19:48:47.163 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T19:48:47.198 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T19:48:47.275 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T19:48:47.915 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T19:48:48.020 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T19:48:49.235 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T19:48:49.318 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T19:48:49.346 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T19:48:49.378 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T19:48:49.691 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T19:48:49.733 INFO:teuthology.orchestra.run.smithi079.stdout: Upgrading : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:48:49.823 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 99/156 2024-04-26T19:48:49.846 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 100/156 2024-04-26T19:48:49.882 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T19:48:50.362 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T19:48:50.394 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:48:50.482 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 103/156 2024-04-26T19:48:50.504 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 104/156 2024-04-26T19:48:52.798 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:48:52.834 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:48:53.377 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 105/156 2024-04-26T19:48:53.421 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:48:53.511 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 106/156 2024-04-26T19:48:53.511 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T19:48:53.511 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T19:48:53.511 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:48:53.538 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:49:05.163 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 107/156 2024-04-26T19:49:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /sys 2024-04-26T19:49:05.163 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /proc 2024-04-26T19:49:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /mnt 2024-04-26T19:49:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /var/tmp 2024-04-26T19:49:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /home 2024-04-26T19:49:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /root 2024-04-26T19:49:05.164 INFO:teuthology.orchestra.run.smithi079.stdout:skipping the directory /tmp 2024-04-26T19:49:05.164 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:05.341 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:49:05.355 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 108/156 2024-04-26T19:49:05.356 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:49:05.356 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T19:49:05.356 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:06.860 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:49:06.886 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 109/156 2024-04-26T19:49:06.886 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:49:06.886 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T19:49:06.886 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:49:06.887 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T19:49:06.887 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:07.111 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:49:07.134 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 110/156 2024-04-26T19:49:07.134 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:49:07.134 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T19:49:07.134 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:49:07.134 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T19:49:07.134 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:07.578 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:49:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 111/156 2024-04-26T19:49:07.601 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:49:07.601 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T19:49:07.601 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:49:07.601 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T19:49:07.601 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:07.700 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T19:49:07.757 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T19:49:07.812 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T19:49:08.006 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T19:49:08.053 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T19:49:08.124 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T19:49:08.212 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:49:08.359 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:49:08.395 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T19:49:08.427 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T19:49:08.455 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T19:49:08.898 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:49:08.928 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 123/156 2024-04-26T19:49:08.960 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 124/156 2024-04-26T19:49:09.022 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 125/156 2024-04-26T19:49:09.051 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:49:10.082 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 126/156 2024-04-26T19:49:10.119 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:49:11.078 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 127/156 2024-04-26T19:49:11.103 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:49:11.203 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 128/156 2024-04-26T19:49:11.365 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 129/156 2024-04-26T19:49:11.388 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:49:11.411 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 130/156 2024-04-26T19:49:11.411 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:49:11.411 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T19:49:11.411 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:49:11.412 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T19:49:11.412 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:11.456 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:49:11.467 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 131/156 2024-04-26T19:49:11.564 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 132/156 2024-04-26T19:49:11.612 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 133/156 2024-04-26T19:49:12.533 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T19:49:12.567 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:49:12.589 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 135/156 2024-04-26T19:49:12.590 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:49:12.590 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T19:49:12.590 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:49:12.590 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T19:49:12.590 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:12.634 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:49:12.656 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3306.gbc612 136/156 2024-04-26T19:49:12.656 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:49:12.656 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T19:49:12.656 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:12.701 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:49:12.724 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 137/156 2024-04-26T19:49:12.724 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:49:12.724 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T19:49:12.724 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:49:12.725 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T19:49:12.725 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:13.027 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:49:13.049 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 138/156 2024-04-26T19:49:13.050 INFO:teuthology.orchestra.run.smithi079.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T19:49:13.050 INFO:teuthology.orchestra.run.smithi079.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T19:49:13.050 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:49:13.050 INFO:teuthology.orchestra.run.smithi079.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T19:49:13.050 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:17.630 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 139/156 2024-04-26T19:49:17.671 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 140/156 2024-04-26T19:49:17.712 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 141/156 2024-04-26T19:49:17.747 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T19:49:17.774 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T19:49:17.956 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 144/156 2024-04-26T19:49:18.003 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T19:49:18.033 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 146/156 2024-04-26T19:49:18.621 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:49:18.666 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:49:18.683 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T19:49:19.506 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 148/156 2024-04-26T19:49:19.594 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T19:49:19.635 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T19:49:19.716 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T19:49:19.757 INFO:teuthology.orchestra.run.smithi079.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T19:49:19.757 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:49:19.849 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T19:49:19.882 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:49:19.882 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:49:19.968 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T19:49:20.039 INFO:teuthology.orchestra.run.smithi079.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:49:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:49:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 1/156 2024-04-26T19:49:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2/156 2024-04-26T19:49:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 3/156 2024-04-26T19:49:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 4/156 2024-04-26T19:49:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3306.gbc612 5/156 2024-04-26T19:49:26.099 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 6/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 7/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 8/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 9/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 10/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 11/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 12/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 13/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 14/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_ 15/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 16/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 17/156 2024-04-26T19:49:26.100 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_6 18/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86 19/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 20/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el 21/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9. 22/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_6 23/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 24/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 25/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 26/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 27/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 28/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 29/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb. 30/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noa 31/156 2024-04-26T19:49:26.101 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.n 32/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc6 33/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el 34/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 35/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.e 36/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 37/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 38/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 39/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T19:49:26.102 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T19:49:26.103 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T19:49:26.104 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T19:49:26.105 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T19:49:26.106 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T19:49:26.107 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T19:49:26.108 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T19:49:26.109 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T19:49:26.110 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 149/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 151/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T19:49:26.111 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T19:49:28.224 INFO:teuthology.orchestra.run.smithi079.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout:Upgraded: 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: librados2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: librbd1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout:Installed: 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.225 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-base-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-grafana-dashboards-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-immutable-object-cache-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mds-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-dashboard-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-modules-core-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mgr-rook-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-mon-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-osd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.226 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-prometheus-alerts-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-radosgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-selinux-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-test-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: ceph-volume-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: cephadm-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-shell-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: cephfs-top-2:19.0.0-3306.gbc612ffb.el9.noarch 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T19:49:28.227 INFO:teuthology.orchestra.run.smithi079.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: libcephfs2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: libcephsqlite-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T19:49:28.228 INFO:teuthology.orchestra.run.smithi079.stdout: librados-devel-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: libradosstriper1-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: librgw2-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T19:49:28.229 INFO:teuthology.orchestra.run.smithi079.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T19:49:28.230 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-argparse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ceph-common-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cephfs-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T19:49:28.231 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T19:49:28.232 INFO:teuthology.orchestra.run.smithi079.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T19:49:28.233 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rados-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rgw-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T19:49:28.234 INFO:teuthology.orchestra.run.smithi079.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-fuse-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.235 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-mirror-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: rbd-nbd-2:19.0.0-3306.gbc612ffb.el9.x86_64 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:28.236 INFO:teuthology.orchestra.run.smithi079.stdout:Complete! 2024-04-26T19:49:28.644 DEBUG:teuthology.parallel:result is None 2024-04-26T19:49:28.645 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:49:28.791 DEBUG:teuthology.orchestra.run.smithi079:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T19:49:28.812 INFO:teuthology.orchestra.run.smithi079.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T19:49:28.813 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T19:49:28.813 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T19:49:28.815 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc612ffb9c1d5bc27536bc917a705262a36d4387 2024-04-26T19:49:29.028 DEBUG:teuthology.orchestra.run.smithi083:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T19:49:29.054 INFO:teuthology.orchestra.run.smithi083.stdout:19.0.0-3306.gbc612ffb.el9 2024-04-26T19:49:29.054 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3306.gbc612ffb.el9 2024-04-26T19:49:29.054 INFO:teuthology.task.install:The correct ceph version 19.0.0-3306.gbc612ffb is installed. 2024-04-26T19:49:29.056 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T19:49:29.056 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:29.056 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T19:49:29.081 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:49:29.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T19:49:29.123 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T19:49:29.124 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:29.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T19:49:29.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T19:49:29.212 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:49:29.212 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T19:49:29.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T19:49:29.305 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T19:49:29.305 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:29.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T19:49:29.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T19:49:29.391 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:49:29.391 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T19:49:29.415 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T19:49:29.478 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T19:49:29.479 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:29.479 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T19:49:29.501 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T19:49:29.564 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:49:29.564 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T19:49:29.588 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T19:49:29.654 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T19:49:29.740 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T19:49:29.741 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 777 /var/log/ceph 2024-04-26T19:49:29.743 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 777 /var/log/ceph 2024-04-26T19:49:29.766 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T19:49:29.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T19:49:29.806 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T19:49:29.829 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T19:49:29.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T19:49:29.869 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T19:49:29.899 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-26T19:49:29.900 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-26T19:49:29.900 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-26T19:49:29.900 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:29.900 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T19:49:29.945 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T19:49:30.060 DEBUG:teuthology.orchestra.run.smithi079:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T19:49:30.082 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:49:30.083 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-26T19:49:30.097 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-26T19:49:30.206 DEBUG:teuthology.orchestra.run.smithi083:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-26T19:49:30.249 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T19:49:30.249 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-26T19:49:30.249 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111', 'branch': 'wip-rishabh-testing-20240426.111959', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/snap_schedule_snapdir}', 'email': None, 'first_in_suite': False, 'job_id': '7675111', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240426.111959'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'kclient': {'snapdirname': '.customsnapkernel', 'syntax': 'v1'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240426.111959', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 816, 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'sleep_before_teardown': 0, 'subset': '166/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240426.111959', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bc612ffb9c1d5bc27536bc917a705262a36d4387/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi079.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBzJLScmthQdf3TjaKOuZ2+yARE0Zi4vrwZkN45ntwzkqt8/gyGQVAFu5Ss00qohimKKK1AiZtzvbB3w3rO8Euo=', 'smithi083.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK1c0p100gtm5EsmevwkWsnBpyVPa5K13Ky32wKej6fx0tGghVSxbcOOS7/YKJznYeffvlzJSUrfHO7SN73k99Y='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc612ffb9c1d5bc27536bc917a705262a36d4387', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_19:30:57', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T19:49:30.250 DEBUG:teuthology.orchestra.run.smithi079:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T19:49:30.252 DEBUG:teuthology.orchestra.run.smithi083:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T19:49:30.297 DEBUG:teuthology.orchestra.run.smithi079:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T19:49:30.306 DEBUG:teuthology.orchestra.run.smithi083:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T19:49:30.361 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:30.362 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T19:49:30.382 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T19:49:30.383 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_1 2024-04-26T19:49:30.437 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T19:49:30.437 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:49:30.437 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T19:49:30.437 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:49:30.437 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:49:30.437 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-04-26 19:49:20.392246020 +0000 2024-04-26T19:49:30.437 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-04-26 19:46:40.668047591 +0000 2024-04-26T19:49:30.437 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-04-26 19:46:40.668047591 +0000 2024-04-26T19:49:30.437 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-04-26 19:46:40.668047591 +0000 2024-04-26T19:49:30.437 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T19:49:30.499 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-04-26T19:49:30.499 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-04-26T19:49:30.499 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000197924 s, 2.6 MB/s 2024-04-26T19:49:30.501 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T19:49:30.556 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_2 2024-04-26T19:49:30.611 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T19:49:30.611 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:49:30.611 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T19:49:30.612 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:49:30.612 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:49:30.612 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-04-26 19:49:20.393245998 +0000 2024-04-26T19:49:30.612 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-04-26 19:46:41.171035581 +0000 2024-04-26T19:49:30.612 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-04-26 19:46:41.171035581 +0000 2024-04-26T19:49:30.612 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-04-26 19:46:41.171035581 +0000 2024-04-26T19:49:30.612 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T19:49:30.673 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-04-26T19:49:30.673 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-04-26T19:49:30.673 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000171065 s, 3.0 MB/s 2024-04-26T19:49:30.674 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T19:49:30.730 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_3 2024-04-26T19:49:30.785 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T19:49:30.786 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:49:30.786 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T19:49:30.786 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:49:30.786 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:49:30.786 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-04-26 19:49:20.393245998 +0000 2024-04-26T19:49:30.786 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-04-26 19:46:41.500027720 +0000 2024-04-26T19:49:30.787 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-04-26 19:46:41.500027720 +0000 2024-04-26T19:49:30.787 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-04-26 19:46:41.500027720 +0000 2024-04-26T19:49:30.787 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T19:49:30.848 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-04-26T19:49:30.848 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-04-26T19:49:30.849 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000196075 s, 2.6 MB/s 2024-04-26T19:49:30.849 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T19:49:30.906 DEBUG:teuthology.orchestra.run.smithi079:> stat /dev/vg_nvme/lv_4 2024-04-26T19:49:30.962 INFO:teuthology.orchestra.run.smithi079.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T19:49:30.962 INFO:teuthology.orchestra.run.smithi079.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:49:30.962 INFO:teuthology.orchestra.run.smithi079.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T19:49:30.962 INFO:teuthology.orchestra.run.smithi079.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:49:30.962 INFO:teuthology.orchestra.run.smithi079.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:49:30.963 INFO:teuthology.orchestra.run.smithi079.stdout:Access: 2024-04-26 19:49:20.394245975 +0000 2024-04-26T19:49:30.963 INFO:teuthology.orchestra.run.smithi079.stdout:Modify: 2024-04-26 19:46:41.903018092 +0000 2024-04-26T19:49:30.963 INFO:teuthology.orchestra.run.smithi079.stdout:Change: 2024-04-26 19:46:41.903018092 +0000 2024-04-26T19:49:30.963 INFO:teuthology.orchestra.run.smithi079.stdout: Birth: 2024-04-26 19:46:41.903018092 +0000 2024-04-26T19:49:30.963 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T19:49:31.024 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records in 2024-04-26T19:49:31.024 INFO:teuthology.orchestra.run.smithi079.stderr:1+0 records out 2024-04-26T19:49:31.024 INFO:teuthology.orchestra.run.smithi079.stderr:512 bytes copied, 0.000181559 s, 2.8 MB/s 2024-04-26T19:49:31.026 DEBUG:teuthology.orchestra.run.smithi079:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T19:49:31.082 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:31.082 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:49:31.082 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T19:49:31.100 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T19:49:31.100 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_1 2024-04-26T19:49:31.155 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T19:49:31.156 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:49:31.156 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T19:49:31.156 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:49:31.156 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:49:31.156 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-26 19:48:32.851931691 +0000 2024-04-26T19:49:31.156 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-26 19:46:42.788849261 +0000 2024-04-26T19:49:31.156 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-26 19:46:42.788849261 +0000 2024-04-26T19:49:31.156 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-26 19:46:42.788849261 +0000 2024-04-26T19:49:31.156 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T19:49:31.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-26T19:49:31.219 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-26T19:49:31.219 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000282721 s, 1.8 MB/s 2024-04-26T19:49:31.220 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T19:49:31.277 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_2 2024-04-26T19:49:31.332 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T19:49:31.332 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:49:31.332 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T19:49:31.332 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:49:31.332 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:49:31.333 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-26 19:48:32.851931691 +0000 2024-04-26T19:49:31.333 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-26 19:46:43.122840826 +0000 2024-04-26T19:49:31.333 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-26 19:46:43.122840826 +0000 2024-04-26T19:49:31.333 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-26 19:46:43.122840826 +0000 2024-04-26T19:49:31.333 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T19:49:31.395 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-26T19:49:31.395 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-26T19:49:31.395 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000249592 s, 2.1 MB/s 2024-04-26T19:49:31.397 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T19:49:31.453 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_3 2024-04-26T19:49:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T19:49:31.508 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:49:31.508 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T19:49:31.508 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:49:31.508 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:49:31.508 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-26 19:48:32.851931691 +0000 2024-04-26T19:49:31.508 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-26 19:46:43.459832317 +0000 2024-04-26T19:49:31.509 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-26 19:46:43.459832317 +0000 2024-04-26T19:49:31.509 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-26 19:46:43.459832317 +0000 2024-04-26T19:49:31.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T19:49:31.572 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-26T19:49:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-26T19:49:31.573 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000198445 s, 2.6 MB/s 2024-04-26T19:49:31.575 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T19:49:31.630 DEBUG:teuthology.orchestra.run.smithi083:> stat /dev/vg_nvme/lv_4 2024-04-26T19:49:31.686 INFO:teuthology.orchestra.run.smithi083.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T19:49:31.686 INFO:teuthology.orchestra.run.smithi083.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T19:49:31.686 INFO:teuthology.orchestra.run.smithi083.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-26T19:49:31.686 INFO:teuthology.orchestra.run.smithi083.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T19:49:31.686 INFO:teuthology.orchestra.run.smithi083.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T19:49:31.686 INFO:teuthology.orchestra.run.smithi083.stdout:Access: 2024-04-26 19:48:32.852931667 +0000 2024-04-26T19:49:31.686 INFO:teuthology.orchestra.run.smithi083.stdout:Modify: 2024-04-26 19:46:43.802823655 +0000 2024-04-26T19:49:31.686 INFO:teuthology.orchestra.run.smithi083.stdout:Change: 2024-04-26 19:46:43.802823655 +0000 2024-04-26T19:49:31.686 INFO:teuthology.orchestra.run.smithi083.stdout: Birth: 2024-04-26 19:46:43.802823655 +0000 2024-04-26T19:49:31.687 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T19:49:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records in 2024-04-26T19:49:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:1+0 records out 2024-04-26T19:49:31.748 INFO:teuthology.orchestra.run.smithi083.stderr:512 bytes copied, 0.000161648 s, 3.2 MB/s 2024-04-26T19:49:31.749 DEBUG:teuthology.orchestra.run.smithi083:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T19:49:31.805 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:31.806 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T19:49:31.806 INFO:tasks.ceph:Generating config... 2024-04-26T19:49:31.808 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-26T19:49:31.808 INFO:tasks.ceph:[client] client snapdir = .customsnapfuse 2024-04-26T19:49:31.808 INFO:tasks.ceph:[client] debug client = 20 2024-04-26T19:49:31.808 INFO:tasks.ceph:[client] debug ms = 1 2024-04-26T19:49:31.808 INFO:tasks.ceph:[client] pid file = 2024-04-26T19:49:31.809 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-26T19:49:31.809 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-26T19:49:31.809 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T19:49:31.809 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T19:49:31.810 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T19:49:31.810 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T19:49:31.810 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T19:49:31.810 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-26T19:49:31.810 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-26T19:49:31.810 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T19:49:31.810 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T19:49:31.810 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-26T19:49:31.810 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-26T19:49:31.810 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T19:49:31.810 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T19:49:31.810 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-26T19:49:31.810 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T19:49:31.810 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T19:49:31.811 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T19:49:31.811 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T19:49:31.811 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T19:49:31.811 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-26T19:49:31.811 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T19:49:31.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T19:49:31.874 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T19:49:31.877 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T19:49:31.916 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T19:49:31.981 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.79'), ('mon.b', '172.21.15.83'), ('mon.c', '[v2:172.21.15.83:3301,v1:172.21.15.83:6790]')] 2024-04-26T19:49:31.982 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.79,172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client snapdir': '.customsnapfuse', 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-26T19:49:31.983 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:31.983 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T19:49:32.037 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.79 --add b 172.21.15.83 --addv c '[v2:172.21.15.83:3301,v1:172.21.15.83:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: generated fsid b372f41f-f07b-4764-9bef-bcff62f9a5d2 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:setting min_mon_release = quincy 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:epoch 0 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:fsid b372f41f-f07b-4764-9bef-bcff62f9a5d2 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:last_changed 2024-04-26T19:49:32.122673+0000 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:created 2024-04-26T19:49:32.122673+0000 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:min_mon_release 17 (quincy) 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:election_strategy: 1 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:0: [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] mon.a 2024-04-26T19:49:32.122 INFO:teuthology.orchestra.run.smithi079.stdout:1: [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] mon.b 2024-04-26T19:49:32.123 INFO:teuthology.orchestra.run.smithi079.stdout:2: [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] mon.c 2024-04-26T19:49:32.123 INFO:teuthology.orchestra.run.smithi079.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T19:49:32.124 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T19:49:32.138 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b372f41f-f07b-4764-9bef-bcff62f9a5d2... 2024-04-26T19:49:32.139 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T19:49:32.181 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T19:49:32.219 INFO:teuthology.orchestra.run.smithi079.stdout:[global] 2024-04-26T19:49:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: chdir = "" 2024-04-26T19:49:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T19:49:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: auth supported = cephx 2024-04-26T19:49:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: filestore xattr use omap = true 2024-04-26T19:49:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: mon clock drift allowed = 1.000 2024-04-26T19:49:32.219 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: osd crush chooseleaf type = 0 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: auth debug = true 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on old message = true 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on bug = true 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: mon pg warn max object skew = 0 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.220 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default size = 2 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow primary affinity = true 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd allow pg remap = true 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on legacy crush tunables = false 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on crush straw calc version zero = false 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on no sortbitwise = false 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on osd down out interval zero = false 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon warn on too few osds = false 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: mon_allow_pool_size_one = true 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: osd default data pool replay window = 5 2024-04-26T19:49:32.221 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: mon allow pool delete = true 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: mon cluster log file level = debug 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: debug asserts on shutdown = true 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: mon health detail to clog = false 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: mon host = "172.21.15.79,172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790]" 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: ms die on skipped message = False 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: fsid = b372f41f-f07b-4764-9bef-bcff62f9a5d2 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout:[osd] 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: osd journal size = 100 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub load threshold = 5.0 2024-04-26T19:49:32.222 INFO:teuthology.orchestra.run.smithi079.stdout: osd scrub max interval = 600 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd mclock profile = high_recovery_ops 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd recover clone overlap = true 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd recovery max chunk = 1048576 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug shutdown = true 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug op order = true 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug verify stray on activate = true 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug trim objects = true 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd open classes on start = true 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug pg log writeout = true 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd deep scrub update digest min age = 30 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.223 INFO:teuthology.orchestra.run.smithi079.stdout: osd map max advance = 10 2024-04-26T19:49:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: journal zero on create = true 2024-04-26T19:49:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: filestore ondisk finisher threads = 3 2024-04-26T19:49:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: filestore apply finisher threads = 3 2024-04-26T19:49:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: bdev debug aio = true 2024-04-26T19:49:32.224 INFO:teuthology.orchestra.run.smithi079.stdout: osd debug misdirected ops = true 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore block size = 96636764160 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: bluestore fsck on mount = True 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluefs = 20 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: debug bluestore = 20 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: debug osd = 20 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: debug rocksdb = 10 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd full ratio = 0.9 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd nearfull ratio = 0.8 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: osd failsafe full ratio = 0.95 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: osd objectstore = bluestore 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: osd op complaint time = 180 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr] 2024-04-26T19:49:32.225 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: debug mgr = 20 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: mgr/telemetry/nag = false 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: debug client = 20 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout:[mon] 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: debug mon = 20 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: debug paxos = 20 2024-04-26T19:49:32.226 INFO:teuthology.orchestra.run.smithi079.stdout: debug auth = 20 2024-04-26T19:49:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: mon data avail warn = 5 2024-04-26T19:49:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: mon mgr mkfs grace = 240 2024-04-26T19:49:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min pgs per osd = 4 2024-04-26T19:49:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd reporter subtree level = osd 2024-04-26T19:49:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: mon osd prime pg temp = true 2024-04-26T19:49:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: mon reweight min bytes per osd = 10 2024-04-26T19:49:32.227 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: # 1m isn't quite enough 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: mon_down_mkfs_grace = 2m 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: mon_warn_on_filestore_osds = false 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: mon down mkfs grace = 300 2024-04-26T19:49:32.228 INFO:teuthology.orchestra.run.smithi079.stdout: mon op complaint time = 120 2024-04-26T19:49:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:49:32.229 INFO:teuthology.orchestra.run.smithi079.stdout:[client] 2024-04-26T19:49:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: rgw cache enabled = true 2024-04-26T19:49:32.229 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable ops log = true 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: rgw enable usage log = true 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: client mount timeout = 600 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: client snapdir = .customsnapfuse 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: debug client = 20 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: pid file = "" 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: rados mon op timeout = 900 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout: rados osd op timeout = 900 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.a] 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.b] 2024-04-26T19:49:32.230 INFO:teuthology.orchestra.run.smithi079.stdout:[mon.c] 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.a] 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.c] 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b] 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.d] 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout:[mds] 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: debug mds = 20 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: debug mds balancer = 20 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: debug ms = 1 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: mds debug frag = True 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: mds debug scatterstat = True 2024-04-26T19:49:32.231 INFO:teuthology.orchestra.run.smithi079.stdout: mds op complaint time = 180 2024-04-26T19:49:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: mds verify scatter = True 2024-04-26T19:49:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: osd op complaint time = 180 2024-04-26T19:49:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: rados mon op timeout = 900 2024-04-26T19:49:32.232 INFO:teuthology.orchestra.run.smithi079.stdout: rados osd op timeout = 900 2024-04-26T19:49:32.233 INFO:teuthology.orchestra.run.smithi083.stdout:[global] 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: chdir = "" 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: auth supported = cephx 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: filestore xattr use omap = true 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: mon clock drift allowed = 1.000 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: osd crush chooseleaf type = 0 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: auth debug = true 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on old message = true 2024-04-26T19:49:32.234 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on bug = true 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon pg warn max object skew = 0 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default size = 2 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow primary affinity = true 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd allow pg remap = true 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on legacy crush tunables = false 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on crush straw calc version zero = false 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on no sortbitwise = false 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on osd down out interval zero = false 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon warn on too few osds = false 2024-04-26T19:49:32.235 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: mon_allow_pool_size_one = true 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: osd default data pool replay window = 5 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: mon allow pool delete = true 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: mon cluster log file level = debug 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: debug asserts on shutdown = true 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: mon health detail to clog = false 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: mon host = "172.21.15.79,172.21.15.83,[v2:172.21.15.83:3301,v1:172.21.15.83:6790]" 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: ms die on skipped message = False 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: fsid = b372f41f-f07b-4764-9bef-bcff62f9a5d2 2024-04-26T19:49:32.236 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout:[osd] 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd journal size = 100 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub load threshold = 5.0 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd scrub max interval = 600 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd mclock profile = high_recovery_ops 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd recover clone overlap = true 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd recovery max chunk = 1048576 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug shutdown = true 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug op order = true 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug verify stray on activate = true 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug trim objects = true 2024-04-26T19:49:32.237 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: osd open classes on start = true 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug pg log writeout = true 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: osd deep scrub update digest min age = 30 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: osd map max advance = 10 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: journal zero on create = true 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: filestore ondisk finisher threads = 3 2024-04-26T19:49:32.238 INFO:teuthology.orchestra.run.smithi083.stdout: filestore apply finisher threads = 3 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: bdev debug aio = true 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: osd debug misdirected ops = true 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore block size = 96636764160 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: bluestore fsck on mount = True 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluefs = 20 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: debug bluestore = 20 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: debug osd = 20 2024-04-26T19:49:32.239 INFO:teuthology.orchestra.run.smithi083.stdout: debug rocksdb = 10 2024-04-26T19:49:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T19:49:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd full ratio = 0.9 2024-04-26T19:49:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd nearfull ratio = 0.8 2024-04-26T19:49:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: osd failsafe full ratio = 0.95 2024-04-26T19:49:32.240 INFO:teuthology.orchestra.run.smithi083.stdout: osd objectstore = bluestore 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr] 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: debug mgr = 20 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: mgr/telemetry/nag = false 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-04-26T19:49:32.241 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout:[mon] 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: debug mon = 20 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: debug paxos = 20 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: debug auth = 20 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: mon data avail warn = 5 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: mon mgr mkfs grace = 240 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min pgs per osd = 4 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd reporter subtree level = osd 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: mon osd prime pg temp = true 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: mon reweight min bytes per osd = 10 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.242 INFO:teuthology.orchestra.run.smithi083.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: # 1m isn't quite enough 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: mon_down_mkfs_grace = 2m 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: mon_warn_on_filestore_osds = false 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: mon down mkfs grace = 300 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: mon op complaint time = 120 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout: 2024-04-26T19:49:32.243 INFO:teuthology.orchestra.run.smithi083.stdout:[client] 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: rgw cache enabled = true 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable ops log = true 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: rgw enable usage log = true 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: client mount timeout = 600 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: client snapdir = .customsnapfuse 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: debug client = 20 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: pid file = "" 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-26T19:49:32.244 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.a] 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.b] 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout:[mon.c] 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout:[mds] 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds = 20 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: debug mds balancer = 20 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: debug ms = 1 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug frag = True 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: mds debug scatterstat = True 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: mds op complaint time = 180 2024-04-26T19:49:32.245 INFO:teuthology.orchestra.run.smithi083.stdout: mds verify scatter = True 2024-04-26T19:49:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: osd op complaint time = 180 2024-04-26T19:49:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: rados mon op timeout = 900 2024-04-26T19:49:32.246 INFO:teuthology.orchestra.run.smithi083.stdout: rados osd op timeout = 900 2024-04-26T19:49:32.246 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T19:49:32.246 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T19:49:32.325 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T19:49:32.326 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:32.326 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T19:49:32.340 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:32.340 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T19:49:32.395 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:49:32.395 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:32.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T19:49:32.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T19:49:32.467 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:49:32.468 DEBUG:teuthology.orchestra.run.smithi079:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:32.523 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:49:32.523 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:49:32.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T19:49:32.523 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T19:49:32.557 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:49:32.558 DEBUG:teuthology.orchestra.run.smithi083:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:32.613 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T19:49:32.613 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T19:49:32.613 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T19:49:32.661 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-26T19:49:32.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T19:49:32.715 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T19:49:32.718 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T19:49:32.719 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T19:49:32.767 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-26T19:49:32.770 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-26T19:49:32.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T19:49:32.882 INFO:teuthology.orchestra.run.smithi079.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-26T19:49:32.885 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-26T19:49:32.907 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T19:49:32.955 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-26T19:49:32.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-26T19:49:32.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T19:49:33.070 INFO:teuthology.orchestra.run.smithi083.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-26T19:49:33.073 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-26T19:49:33.097 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T19:49:33.097 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T19:49:33.136 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-26T19:49:33.148 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-26T19:49:33.228 INFO:teuthology.orchestra.run.smithi079.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-26T19:49:33.240 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T19:49:33.279 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-26T19:49:33.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-26T19:49:33.369 INFO:teuthology.orchestra.run.smithi083.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-26T19:49:33.381 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T19:49:33.381 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi079.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi083.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-26T19:49:33.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T19:49:33.405 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:33.405 INFO:tasks.ceph:role: osd.0 2024-04-26T19:49:33.405 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:49:33.406 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T19:49:33.476 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:49:33.476 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:49:33.477 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:49:33.477 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:49:33.477 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:49:33.477 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:49:33.477 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:49:33.477 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:49:33.477 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:49:33.477 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:49:33.991 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-04-26T19:49:33.993 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-04-26T19:49:33.994 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T19:49:34.129 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T19:49:34.161 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T19:49:34.226 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:34.226 INFO:tasks.ceph:role: osd.1 2024-04-26T19:49:34.226 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:49:34.227 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T19:49:34.298 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:49:34.298 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:49:34.298 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:49:34.298 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:49:34.298 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:49:34.298 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:49:34.298 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:49:34.298 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:49:34.298 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:49:34.299 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:49:34.748 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-04-26T19:49:34.751 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-04-26T19:49:34.751 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T19:49:34.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T19:49:34.872 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T19:49:34.937 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:34.937 INFO:tasks.ceph:role: osd.2 2024-04-26T19:49:34.937 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:49:34.938 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:49:35.009 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:49:35.505 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-04-26T19:49:35.507 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-04-26T19:49:35.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T19:49:35.638 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T19:49:35.665 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-26T19:49:35.728 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:35.728 INFO:tasks.ceph:role: osd.3 2024-04-26T19:49:35.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:49:35.729 DEBUG:teuthology.orchestra.run.smithi079:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:49:35.799 INFO:teuthology.orchestra.run.smithi079.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:49:36.304 INFO:teuthology.orchestra.run.smithi079.stdout:Discarding blocks...Done. 2024-04-26T19:49:36.305 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi079.front.sepia.ceph.com -o noatime 2024-04-26T19:49:36.306 DEBUG:teuthology.orchestra.run.smithi079:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-26T19:49:36.422 DEBUG:teuthology.orchestra.run.smithi079:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-26T19:49:36.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:36.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:36.542+0000 7fb3da663640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T19:49:36.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:36.542+0000 7fb3da663640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T19:49:36.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:36.542+0000 7fb3da663640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T19:49:36.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:36.542+0000 7fb3da663640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T19:49:36.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:36.542+0000 7fb3da663640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T19:49:36.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:36.543+0000 7fb3da663640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T19:49:39.395 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T19:49:39.423 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:39.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:39.511+0000 7fa2d7350640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T19:49:39.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:39.511+0000 7fa2d7350640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T19:49:39.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:39.511+0000 7fa2d7350640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T19:49:39.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:39.511+0000 7fa2d7350640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T19:49:39.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:39.511+0000 7fa2d7350640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T19:49:39.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:39.512+0000 7fa2d7350640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T19:49:42.357 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T19:49:42.385 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:42.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:42.472+0000 7f3b96425640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T19:49:42.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:42.472+0000 7f3b96425640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T19:49:42.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:42.472+0000 7f3b96425640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T19:49:42.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:42.472+0000 7f3b96425640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T19:49:42.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:42.472+0000 7f3b96425640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T19:49:42.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:42.473+0000 7f3b96425640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T19:49:45.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T19:49:45.355 DEBUG:teuthology.orchestra.run.smithi079:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:45.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:45.443+0000 7f98c0b53640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-26T19:49:45.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:45.443+0000 7f98c0b53640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-26T19:49:45.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:45.443+0000 7f98c0b53640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T19:49:45.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:45.443+0000 7f98c0b53640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T19:49:45.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:45.443+0000 7f98c0b53640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-26T19:49:45.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:49:45.444+0000 7f98c0b53640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-26T19:49:48.300 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T19:49:48.328 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-26T19:49:48.358 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:48.358 INFO:tasks.ceph:role: osd.4 2024-04-26T19:49:48.358 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:49:48.358 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T19:49:48.432 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:49:48.432 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:49:48.432 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:49:48.432 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:49:48.432 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:49:48.432 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:49:48.432 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:49:48.432 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:49:48.432 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:49:48.433 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:49:48.947 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-26T19:49:48.949 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-26T19:49:48.949 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-26T19:49:49.076 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-26T19:49:49.110 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-26T19:49:49.176 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:49.176 INFO:tasks.ceph:role: osd.5 2024-04-26T19:49:49.177 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:49:49.177 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:49:49.250 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:49:49.717 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-26T19:49:49.719 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-26T19:49:49.720 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-26T19:49:49.814 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-26T19:49:49.844 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-26T19:49:49.909 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:49.910 INFO:tasks.ceph:role: osd.6 2024-04-26T19:49:49.910 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:49:49.910 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T19:49:49.982 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:49:49.983 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:49:50.471 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-26T19:49:50.473 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-26T19:49:50.473 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-26T19:49:50.594 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-26T19:49:50.627 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-26T19:49:50.693 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-26T19:49:50.693 INFO:tasks.ceph:role: osd.7 2024-04-26T19:49:50.693 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:49:50.693 DEBUG:teuthology.orchestra.run.smithi083:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-26T19:49:50.765 INFO:teuthology.orchestra.run.smithi083.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T19:49:50.766 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T19:49:50.766 INFO:teuthology.orchestra.run.smithi083.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T19:49:50.766 INFO:teuthology.orchestra.run.smithi083.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T19:49:50.766 INFO:teuthology.orchestra.run.smithi083.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T19:49:50.766 INFO:teuthology.orchestra.run.smithi083.stdout: = sunit=0 swidth=0 blks 2024-04-26T19:49:50.766 INFO:teuthology.orchestra.run.smithi083.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T19:49:50.766 INFO:teuthology.orchestra.run.smithi083.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T19:49:50.766 INFO:teuthology.orchestra.run.smithi083.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T19:49:50.766 INFO:teuthology.orchestra.run.smithi083.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T19:49:51.288 INFO:teuthology.orchestra.run.smithi083.stdout:Discarding blocks...Done. 2024-04-26T19:49:51.291 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi083.front.sepia.ceph.com -o noatime 2024-04-26T19:49:51.291 DEBUG:teuthology.orchestra.run.smithi083:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-26T19:49:51.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-26T19:49:51.481 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:51.582 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:51.581+0000 7f7842615640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-26T19:49:51.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:51.581+0000 7f7842615640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-26T19:49:51.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:51.581+0000 7f7842615640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T19:49:51.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:51.581+0000 7f7842615640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T19:49:51.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:51.581+0000 7f7842615640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-26T19:49:51.583 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:51.582+0000 7f7842615640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-26T19:49:54.431 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T19:49:54.459 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:54.547 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:54.546+0000 7fbced957640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-26T19:49:54.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:54.546+0000 7fbced957640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-26T19:49:54.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:54.546+0000 7fbced957640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T19:49:54.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:54.546+0000 7fbced957640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T19:49:54.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:54.546+0000 7fbced957640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-26T19:49:54.548 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:54.547+0000 7fbced957640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-26T19:49:57.393 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T19:49:57.418 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:49:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:57.503+0000 7f17e91c4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-26T19:49:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:57.504+0000 7f17e91c4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-26T19:49:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:57.504+0000 7f17e91c4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T19:49:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:57.504+0000 7f17e91c4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T19:49:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:57.504+0000 7f17e91c4640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-26T19:49:57.505 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:49:57.504+0000 7f17e91c4640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-26T19:50:00.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:50:00.254 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:50:00.364 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-26T19:50:00.390 DEBUG:teuthology.orchestra.run.smithi083:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:50:00.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:00.474+0000 7f0aa1a27640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-26T19:50:00.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:00.475+0000 7f0aa1a27640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-26T19:50:00.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:00.475+0000 7f0aa1a27640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T19:50:00.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:00.475+0000 7f0aa1a27640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T19:50:00.476 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:00.475+0000 7f0aa1a27640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-26T19:50:00.477 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:00.475+0000 7f0aa1a27640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-26T19:50:03.324 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-26T19:50:03.351 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T19:50:03.352 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:03.352 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-26T19:50:03.381 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:03.381 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-26T19:50:03.449 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:03.449 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-26T19:50:03.516 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:03.516 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T19:50:03.582 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:03.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T19:50:03.647 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:03.647 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T19:50:03.712 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:03.712 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-26T19:50:03.776 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:03.776 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T19:50:03.805 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:03.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-26T19:50:03.872 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:03.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-26T19:50:03.942 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:03.942 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-26T19:50:04.007 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:04.008 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-26T19:50:04.073 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:04.074 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-26T19:50:04.139 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:04.139 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-26T19:50:04.204 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:04.204 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-26T19:50:04.222 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:04.222 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-26T19:50:04.279 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:04.280 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-26T19:50:04.295 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:04.295 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-26T19:50:04.351 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T19:50:04.351 DEBUG:teuthology.orchestra.run.smithi079:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T19:50:04.354 DEBUG:teuthology.orchestra.run.smithi083:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T19:50:04.394 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.y] 2024-04-26T19:50:04.394 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMBSxm2bR2JxAAONpqvh+RpLjBgDu2rjiv+Q== 2024-04-26T19:50:04.394 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.a] 2024-04-26T19:50:04.394 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMBSxmKKzNLRAATTamPLUKZz4plkGBTiyOjw== 2024-04-26T19:50:04.394 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.c] 2024-04-26T19:50:04.394 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMBSxm5rCiNBAAxsM3A75gmGA9hb2dlnJY+Q== 2024-04-26T19:50:04.394 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.0] 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBQBSxmUiloIBAAxns7tzts02YP2BVjmjJDyg== 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.1] 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBTBSxm0GCNHhAAhYZJUvZc2fhkt4UNYgb/7Q== 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.2] 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBWBSxmCg42HBAAo+vEqWMyOiC632Saxz/M8Q== 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.3] 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBZBSxmPx17GhAAYYowgL/L38fF57nmjGtAAw== 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout:[mgr.x] 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMBSxmzY+rKhAAqr9zc68jOGYvhstf+AvDXQ== 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.b] 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBMBSxmOMv9OBAAMfVcb1lc/xGbELkGngAmzw== 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout:[mds.d] 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBNBSxm+y5EBBAAiYAmnzhCCNr2PJ7N7MbH+A== 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.4] 2024-04-26T19:50:04.395 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBfBSxmN3i/IhAA4ctD7r7jlwNMcTKtjc9RWg== 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.5] 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBiBSxm5tCnIBAA4uOkO81p3oPAAHhc1iAz8Q== 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.6] 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBlBSxmfZohHhAAxQw7mbsZ2WNlFq7Dt9MODQ== 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout:[osd.7] 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBoBSxmkSVlHBAAfzDuWJJsYr8I4g5we5gQKQ== 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBNBSxmVxwrCBAA6mt2hQ7ZI0Y9b09Qqm6RfQ== 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout:[client.2] 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBNBSxm1rmfDRAAL9yuQwmwQGr1wKj6nBaXNQ== 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout:[client.1] 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBNBSxmeROwEBAAc6r+x9PwvArMxK3fQGtPiw== 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout:[client.3] 2024-04-26T19:50:04.396 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBNBSxmi74TFhAA2qTq35nlb+CmAEGuhpIk+Q== 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.y] 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBMBSxm2bR2JxAAONpqvh+RpLjBgDu2rjiv+Q== 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.a] 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBMBSxmKKzNLRAATTamPLUKZz4plkGBTiyOjw== 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.c] 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBMBSxm5rCiNBAAxsM3A75gmGA9hb2dlnJY+Q== 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.0] 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBQBSxmUiloIBAAxns7tzts02YP2BVjmjJDyg== 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.1] 2024-04-26T19:50:04.413 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBTBSxm0GCNHhAAhYZJUvZc2fhkt4UNYgb/7Q== 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.2] 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBWBSxmCg42HBAAo+vEqWMyOiC632Saxz/M8Q== 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.3] 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBZBSxmPx17GhAAYYowgL/L38fF57nmjGtAAw== 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout:[mgr.x] 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBMBSxmzY+rKhAAqr9zc68jOGYvhstf+AvDXQ== 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.b] 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBMBSxmOMv9OBAAMfVcb1lc/xGbELkGngAmzw== 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout:[mds.d] 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBNBSxm+y5EBBAAiYAmnzhCCNr2PJ7N7MbH+A== 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.4] 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBfBSxmN3i/IhAA4ctD7r7jlwNMcTKtjc9RWg== 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.5] 2024-04-26T19:50:04.414 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBiBSxm5tCnIBAA4uOkO81p3oPAAHhc1iAz8Q== 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.6] 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBlBSxmfZohHhAAxQw7mbsZ2WNlFq7Dt9MODQ== 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout:[osd.7] 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBoBSxmkSVlHBAAfzDuWJJsYr8I4g5we5gQKQ== 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout:[client.0] 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBNBSxmVxwrCBAA6mt2hQ7ZI0Y9b09Qqm6RfQ== 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout:[client.2] 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBNBSxm1rmfDRAAL9yuQwmwQGr1wKj6nBaXNQ== 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout:[client.1] 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBNBSxmeROwEBAAc6r+x9PwvArMxK3fQGtPiw== 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout:[client.3] 2024-04-26T19:50:04.415 INFO:teuthology.orchestra.run.smithi083.stdout: key = AQBNBSxmi74TFhAA2qTq35nlb+CmAEGuhpIk+Q== 2024-04-26T19:50:04.416 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T19:50:04.440 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T19:50:04.507 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:50:04.509 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:50:04.561 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:50:04.563 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:50:04.615 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:04.617 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:04.666 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:04.668 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:04.716 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:04.719 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:04.768 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:04.771 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:04.819 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T19:50:04.821 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T19:50:04.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:50:04.872 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:50:04.923 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:50:04.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-26T19:50:04.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:04.979 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:05.029 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:05.031 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:05.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:05.082 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:05.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:05.135 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T19:50:05.184 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:50:05.188 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:50:05.240 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:50:05.242 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:50:05.293 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:50:05.296 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:50:05.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:50:05.349 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-26T19:50:05.401 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T19:50:05.401 DEBUG:teuthology.orchestra.run.smithi079:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T19:50:05.425 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T19:50:05.520 DEBUG:teuthology.orchestra.run.smithi079:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T19:50:05.545 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T19:50:05.568 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T19:50:05.664 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T19:50:05.688 DEBUG:teuthology.orchestra.run.smithi083:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T19:50:05.751 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T19:50:05.846 DEBUG:teuthology.orchestra.run.smithi083:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T19:50:05.869 DEBUG:teuthology.orchestra.run.smithi079:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:50:05.871 DEBUG:teuthology.orchestra.run.smithi083:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T19:50:05.923 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T19:50:05.924 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T19:50:05.924 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T19:50:05.926 INFO:tasks.ceph.mon.a:Started 2024-04-26T19:50:05.926 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T19:50:05.926 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T19:50:05.966 INFO:tasks.ceph.mon.b:Started 2024-04-26T19:50:05.966 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T19:50:05.967 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T19:50:05.968 INFO:tasks.ceph.mon.c:Started 2024-04-26T19:50:05.968 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T19:50:05.969 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-26T19:50:05.969 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-26T19:50:05.971 INFO:tasks.ceph.mgr.y:Started 2024-04-26T19:50:05.971 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T19:50:05.971 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T19:50:05.973 INFO:tasks.ceph.mgr.x:Started 2024-04-26T19:50:05.974 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T19:50:05.974 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T19:50:06.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.070+0000 7f07f73c1640 1 Processor -- start 2024-04-26T19:50:06.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.070+0000 7f07f73c1640 1 -- start start 2024-04-26T19:50:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f73c1640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f07f01421e0 0x7f07f0160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f73c1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 0x7f07f006f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f73c1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07f0144b50 con 0x7f07f006f5e0 2024-04-26T19:50:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f73c1640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f07f00b0e90 con 0x7f07f01421e0 2024-04-26T19:50:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f73c1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f07f0144e30 con 0x7f07f0160690 2024-04-26T19:50:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f515d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f07f01421e0 0x7f07f0160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f595e640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f07f006f5e0 0x7f07f0162fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:39796/0 (socket says 172.21.15.79:39796) 2024-04-26T19:50:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f595e640 1 -- 172.21.15.79:0/1566471170 learned_addr learned my addr 172.21.15.79:0/1566471170 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:06.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f515d640 1 -- 172.21.15.79:0/1566471170 >> v2:172.21.15.79:3300/0 conn(0x7f07f01421e0 msgr2=0x7f07f0160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-26T19:50:06.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f515d640 1 -- 172.21.15.79:0/1566471170 >> v2:172.21.15.79:3300/0 conn(0x7f07f01421e0 msgr2=0x7f07f0160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T19:50:06.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f515d640 1 --2- 172.21.15.79:0/1566471170 >> v2:172.21.15.79:3300/0 conn(0x7f07f01421e0 0x7f07f0160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T19:50:06.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f515d640 1 --2- 172.21.15.79:0/1566471170 >> v2:172.21.15.79:3300/0 conn(0x7f07f01421e0 0x7f07f0160150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:50:06.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.071+0000 7f07f495c640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 0x7f07f006f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.072+0000 7f07f495c640 1 -- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 msgr2=0x7f07f006f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-26T19:50:06.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.072+0000 7f07f495c640 1 -- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 msgr2=0x7f07f006f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-26T19:50:06.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.072+0000 7f07f495c640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 0x7f07f006f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T19:50:06.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.072+0000 7f07f495c640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 0x7f07f006f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:50:06.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2774710805 0 0) 0x7f07f0144b50 con 0x7f07f006f5e0 2024-04-26T19:50:06.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07dc003520 con 0x7f07f006f5e0 2024-04-26T19:50:06.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3339700997 0 0) 0x7f07dc003520 con 0x7f07f006f5e0 2024-04-26T19:50:06.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 msgr2=0x7f07f006f0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:50:06.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 0x7f07f006f0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 >> v2:172.21.15.79:3300/0 conn(0x7f07f01421e0 msgr2=0x7f07f0160150 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:50:06.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 --2- 172.21.15.79:0/1566471170 >> v2:172.21.15.79:3300/0 conn(0x7f07f01421e0 0x7f07f0160150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f0145110 con 0x7f07f006f5e0 2024-04-26T19:50:06.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f07e80019f0 con 0x7f07f006f5e0 2024-04-26T19:50:06.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 >> v1:172.21.15.79:6789/0 conn(0x7f07f006f5e0 legacy=0x7f07f0162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07dc0039b0 0x7f07dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 0x7f07dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07dc004d40 0x7f07dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f07f00b0e90 con 0x7f07dc0039b0 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07f0144e30 con 0x7f07dc004d40 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc00baf0 con 0x7f07f0160690 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07f515d640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07dc0039b0 0x7f07dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07f595e640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07dc004d40 0x7f07dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.235+0000 7f07f495c640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 0x7f07dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.236+0000 7f07f515d640 1 -- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 msgr2=0x7f07dc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.236+0000 7f07f515d640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f0160690 0x7f07dc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.236+0000 7f07f515d640 1 -- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07dc004d40 msgr2=0x7f07dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.236+0000 7f07f515d640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07dc004d40 0x7f07dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.236+0000 7f07f515d640 1 -- 172.21.15.79:0/1566471170 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc003520 con 0x7f07dc0039b0 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.236+0000 7f07f515d640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07dc0039b0 0x7f07dc003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f07e000b9e0 tx=0x7f07dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95222f289a05bfb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:06.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e0030070 con 0x7f07dc0039b0 2024-04-26T19:50:06.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07e002adb0 con 0x7f07dc0039b0 2024-04-26T19:50:06.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07e7fff640 1 -- 172.21.15.79:0/1566471170 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07e000b5e0 con 0x7f07dc0039b0 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07f73c1640 1 -- 172.21.15.79:0/1566471170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07dc0039b0 msgr2=0x7f07dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07f73c1640 1 --2- 172.21.15.79:0/1566471170 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07dc0039b0 0x7f07dc003dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f07e000b9e0 tx=0x7f07dc005830 comp rx=0 tx=0).stop 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07f595e640 1 -- 172.21.15.79:0/1566471170 reap_dead start 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07f73c1640 1 -- 172.21.15.79:0/1566471170 shutdown_connections 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07f73c1640 1 -- 172.21.15.79:0/1566471170 >> 172.21.15.79:0/1566471170 conn(0x7f07f0068860 msgr2=0x7f07f00b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07f73c1640 1 -- 172.21.15.79:0/1566471170 shutdown_connections 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.237+0000 7f07f73c1640 1 -- 172.21.15.79:0/1566471170 wait complete. 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.239+0000 7f07f73c1640 1 Processor -- start 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.239+0000 7f07f73c1640 1 -- start start 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f73c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07dc0039b0 0x7f07f0140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f73c1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f01638b0 0x7f07f01626d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f73c1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f00cdff0 0x7f07f015db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f73c1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f07f015fd80 con 0x7f07f01638b0 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f73c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07f015fc00 con 0x7f07dc0039b0 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f73c1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f07f01425c0 con 0x7f07f00cdff0 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f495c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f01638b0 0x7f07f01626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f495c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f01638b0 0x7f07f01626d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:48450/0 (socket says 172.21.15.79:48450) 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f515d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07dc0039b0 0x7f07f0140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f495c640 1 -- 172.21.15.79:0/3973413204 learned_addr learned my addr 172.21.15.79:0/3973413204 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.240+0000 7f07f595e640 1 --2- 172.21.15.79:0/3973413204 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f00cdff0 0x7f07f015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07f495c640 1 -- 172.21.15.79:0/3973413204 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f00cdff0 msgr2=0x7f07f015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07f495c640 1 --2- 172.21.15.79:0/3973413204 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f00cdff0 0x7f07f015db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07f495c640 1 -- 172.21.15.79:0/3973413204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07dc0039b0 msgr2=0x7f07f0140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07f495c640 1 --2- 172.21.15.79:0/3973413204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07dc0039b0 0x7f07f0140b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07f495c640 1 -- 172.21.15.79:0/3973413204 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07f00ce840 con 0x7f07f01638b0 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07f595e640 1 --2- 172.21.15.79:0/3973413204 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f00cdff0 0x7f07f015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07f495c640 1 --2- 172.21.15.79:0/3973413204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f01638b0 0x7f07f01626d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0108d0 tx=0x7f07ec010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07e5ffb640 1 -- 172.21.15.79:0/3973413204 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec01a020 con 0x7f07f01638b0 2024-04-26T19:50:06.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07e5ffb640 1 -- 172.21.15.79:0/3973413204 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ec00e9c0 con 0x7f07f01638b0 2024-04-26T19:50:06.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07e5ffb640 1 -- 172.21.15.79:0/3973413204 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ec00ed90 con 0x7f07f01638b0 2024-04-26T19:50:06.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07f73c1640 1 -- 172.21.15.79:0/3973413204 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07f0161bf0 con 0x7f07f01638b0 2024-04-26T19:50:06.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07f73c1640 1 -- 172.21.15.79:0/3973413204 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07f0162c10 con 0x7f07f01638b0 2024-04-26T19:50:06.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07e5ffb640 1 -- 172.21.15.79:0/3973413204 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f07ec01d020 con 0x7f07f01638b0 2024-04-26T19:50:06.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.241+0000 7f07e5ffb640 1 -- 172.21.15.79:0/3973413204 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f07ec014910 con 0x7f07f01638b0 2024-04-26T19:50:06.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.242+0000 7f07f73c1640 1 -- 172.21.15.79:0/3973413204 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07f00506c0 con 0x7f07f01638b0 2024-04-26T19:50:06.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.246+0000 7f07e5ffb640 1 -- 172.21.15.79:0/3973413204 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f07ec024020 con 0x7f07f01638b0 2024-04-26T19:50:06.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.312+0000 7f07f73c1640 1 -- 172.21.15.79:0/3973413204 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f07f0073d90 con 0x7f07f01638b0 2024-04-26T19:50:06.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.314+0000 7f07e5ffb640 1 -- 172.21.15.79:0/3973413204 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f07ec014430 con 0x7f07f01638b0 2024-04-26T19:50:06.314 INFO:teuthology.orchestra.run.smithi079.stderr:adjusted tunables profile to default 2024-04-26T19:50:06.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.315+0000 7f07f73c1640 1 -- 172.21.15.79:0/3973413204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f01638b0 msgr2=0x7f07f01626d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.315+0000 7f07f73c1640 1 --2- 172.21.15.79:0/3973413204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f01638b0 0x7f07f01626d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f07ec0108d0 tx=0x7f07ec010d90 comp rx=0 tx=0).stop 2024-04-26T19:50:06.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.315+0000 7f07f73c1640 1 -- 172.21.15.79:0/3973413204 shutdown_connections 2024-04-26T19:50:06.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.315+0000 7f07f73c1640 1 --2- 172.21.15.79:0/3973413204 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07f00cdff0 0x7f07f015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.315+0000 7f07f73c1640 1 --2- 172.21.15.79:0/3973413204 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07f01638b0 0x7f07f01626d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.315+0000 7f07f73c1640 1 --2- 172.21.15.79:0/3973413204 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07dc0039b0 0x7f07f0140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.315+0000 7f07f73c1640 1 -- 172.21.15.79:0/3973413204 >> 172.21.15.79:0/3973413204 conn(0x7f07f0068860 msgr2=0x7f07f00b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:06.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.315+0000 7f07f73c1640 1 -- 172.21.15.79:0/3973413204 shutdown_connections 2024-04-26T19:50:06.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:06.315+0000 7f07f73c1640 1 -- 172.21.15.79:0/3973413204 wait complete. 2024-04-26T19:50:06.326 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T19:50:06.326 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T19:50:06.326 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:06.326 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T19:50:06.343 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:06.342+0000 7fb7d1810040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:50:06.348 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:06.347+0000 7fdb97d21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:50:06.350 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:06.350 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T19:50:06.414 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:06.414 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T19:50:06.477 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:06.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-26T19:50:06.493 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:06.491+0000 7fdb97d21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:50:06.512 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:06.511+0000 7fb7d1810040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:50:06.534 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:06.534 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-26T19:50:06.546 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:06.545+0000 7fdb97d21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:50:06.558 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:06.558 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-26T19:50:06.567 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:06.567+0000 7fb7d1810040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:50:06.623 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:06.623 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-26T19:50:06.686 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:50:06.686 DEBUG:teuthology.orchestra.run.smithi083:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-26T19:50:06.749 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 58c5c8de-2274-4989-bf9c-1fe2258a4928 0 2024-04-26T19:50:06.801 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:06.799+0000 7fb7d1810040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:50:06.808 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:06.807+0000 7fdb97d21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:50:06.844 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:06.844+0000 7fb7d1810040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:50:06.856 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:06.854+0000 7fdb97d21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:50:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.885+0000 7f470e738640 1 Processor -- start 2024-04-26T19:50:06.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.885+0000 7f470e738640 1 -- start start 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f470e738640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47081421e0 0x7f4708160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f470e738640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f470806f7b0 0x7f47080afe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f470e738640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4708144b50 con 0x7f4708160690 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f470e738640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4708144e30 con 0x7f47081421e0 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f470e738640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4708145110 con 0x7f470806f7b0 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f470ccd5640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f470806f7b0 0x7f47080afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f4707fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f47081421e0 0x7f4708160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f470ccd5640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f470806f7b0 0x7f47080afe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.83:34632/0 (socket says 172.21.15.83:34632) 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f470ccd5640 1 -- 172.21.15.83:0/1658605115 learned_addr learned my addr 172.21.15.83:0/1658605115 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f4707fff640 1 -- 172.21.15.83:0/1658605115 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f470806f7b0 msgr2=0x7f47080afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f4707fff640 1 --2- 172.21.15.83:0/1658605115 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f470806f7b0 0x7f47080afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f4707fff640 1 -- 172.21.15.83:0/1658605115 >> v1:172.21.15.83:6789/0 conn(0x7f4708160690 legacy=0x7f470806f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f4706ffd640 1 -- 172.21.15.83:0/1658605115 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 597410899 0 0) 0x7f4708144b50 con 0x7f4708160690 2024-04-26T19:50:06.888 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f4707fff640 1 -- 172.21.15.83:0/1658605115 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47081453f0 con 0x7f47081421e0 2024-04-26T19:50:06.889 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.886+0000 7f470ccd5640 1 --2- 172.21.15.83:0/1658605115 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f470806f7b0 0x7f47080afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.887+0000 7f4707fff640 1 --2- 172.21.15.83:0/1658605115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47081421e0 0x7f4708160150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0097a0 tx=0x7f46fc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aee05ab55b525d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.887+0000 7f4706ffd640 1 -- 172.21.15.83:0/1658605115 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc037070 con 0x7f47081421e0 2024-04-26T19:50:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.887+0000 7f4706ffd640 1 -- 172.21.15.83:0/1658605115 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc032440 con 0x7f47081421e0 2024-04-26T19:50:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.887+0000 7f4706ffd640 1 -- 172.21.15.83:0/1658605115 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc032810 con 0x7f47081421e0 2024-04-26T19:50:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.888+0000 7f470e738640 1 -- 172.21.15.83:0/1658605115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47081421e0 msgr2=0x7f4708160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.888+0000 7f470e738640 1 --2- 172.21.15.83:0/1658605115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47081421e0 0x7f4708160150 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f46fc0097a0 tx=0x7f46fc02a180 comp rx=0 tx=0).stop 2024-04-26T19:50:06.890 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.888+0000 7f470e738640 1 -- 172.21.15.83:0/1658605115 shutdown_connections 2024-04-26T19:50:06.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.888+0000 7f470e738640 1 --2- 172.21.15.83:0/1658605115 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f470806f7b0 0x7f47080afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.888+0000 7f470e738640 1 --2- 172.21.15.83:0/1658605115 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f47081421e0 0x7f4708160150 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.888+0000 7f470e738640 1 -- 172.21.15.83:0/1658605115 >> 172.21.15.83:0/1658605115 conn(0x7f4708068860 msgr2=0x7f47080b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:06.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.888+0000 7f470e738640 1 -- 172.21.15.83:0/1658605115 shutdown_connections 2024-04-26T19:50:06.891 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.888+0000 7f470e738640 1 -- 172.21.15.83:0/1658605115 wait complete. 2024-04-26T19:50:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.889+0000 7f470e738640 1 Processor -- start 2024-04-26T19:50:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.889+0000 7f470e738640 1 -- start start 2024-04-26T19:50:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470e738640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f470806f7b0 0x7f4708163430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470e738640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f47081421e0 0x7f47080d0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470e738640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4708160690 0x7f470813b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470e738640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4708142740 con 0x7f470806f7b0 2024-04-26T19:50:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470e738640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f47081425c0 con 0x7f4708160690 2024-04-26T19:50:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470e738640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4708071850 con 0x7f47081421e0 2024-04-26T19:50:06.894 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470ccd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4708160690 0x7f470813b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f47077fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f47081421e0 0x7f47080d0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470ccd5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4708160690 0x7f470813b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44280/0 (socket says 172.21.15.83:44280) 2024-04-26T19:50:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470ccd5640 1 -- 172.21.15.83:0/3755027335 learned_addr learned my addr 172.21.15.83:0/3755027335 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:06.895 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f4707fff640 1 --2- 172.21.15.83:0/3755027335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f470806f7b0 0x7f4708163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470ccd5640 1 -- 172.21.15.83:0/3755027335 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f47081421e0 msgr2=0x7f47080d0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.890+0000 7f470ccd5640 1 --2- 172.21.15.83:0/3755027335 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f47081421e0 0x7f47080d0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f470ccd5640 1 -- 172.21.15.83:0/3755027335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f470806f7b0 msgr2=0x7f4708163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f470ccd5640 1 --2- 172.21.15.83:0/3755027335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f470806f7b0 0x7f4708163430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f470ccd5640 1 -- 172.21.15.83:0/3755027335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47080cefc0 con 0x7f4708160690 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f47077fe640 1 --2- 172.21.15.83:0/3755027335 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f47081421e0 0x7f47080d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f470ccd5640 1 --2- 172.21.15.83:0/3755027335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4708160690 0x7f470813b910 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f470000efc0 tx=0x7f470000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f4704ff9640 1 -- 172.21.15.83:0/3755027335 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470000cde0 con 0x7f4708160690 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f4704ff9640 1 -- 172.21.15.83:0/3755027335 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4700003990 con 0x7f4708160690 2024-04-26T19:50:06.896 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f470e738640 1 -- 172.21.15.83:0/3755027335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47080cf2a0 con 0x7f4708160690 2024-04-26T19:50:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f4704ff9640 1 -- 172.21.15.83:0/3755027335 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700003d80 con 0x7f4708160690 2024-04-26T19:50:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f470e738640 1 -- 172.21.15.83:0/3755027335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47080cf860 con 0x7f4708160690 2024-04-26T19:50:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f4704ff9640 1 -- 172.21.15.83:0/3755027335 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4700012050 con 0x7f4708160690 2024-04-26T19:50:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.891+0000 7f4704ff9640 1 -- 172.21.15.83:0/3755027335 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f4700014820 con 0x7f4708160690 2024-04-26T19:50:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.892+0000 7f470e738640 1 -- 172.21.15.83:0/3755027335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47080506c0 con 0x7f4708160690 2024-04-26T19:50:06.897 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.894+0000 7f4704ff9640 1 -- 172.21.15.83:0/3755027335 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4700010040 con 0x7f4708160690 2024-04-26T19:50:06.967 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.966+0000 7f470e738640 1 -- 172.21.15.83:0/3755027335 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "58c5c8de-2274-4989-bf9c-1fe2258a4928", "id": 0} v 0) -- 0x7f4708073ce0 con 0x7f4708160690 2024-04-26T19:50:06.970 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.969+0000 7f4704ff9640 1 -- 172.21.15.83:0/3755027335 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "58c5c8de-2274-4989-bf9c-1fe2258a4928", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f4700014a70 con 0x7f4708160690 2024-04-26T19:50:06.970 INFO:teuthology.orchestra.run.smithi083.stdout:0 2024-04-26T19:50:06.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.970+0000 7f470e738640 1 -- 172.21.15.83:0/3755027335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4708160690 msgr2=0x7f470813b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:06.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.970+0000 7f470e738640 1 --2- 172.21.15.83:0/3755027335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4708160690 0x7f470813b910 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f470000efc0 tx=0x7f470000c490 comp rx=0 tx=0).stop 2024-04-26T19:50:06.971 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.970+0000 7f470e738640 1 -- 172.21.15.83:0/3755027335 shutdown_connections 2024-04-26T19:50:06.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.970+0000 7f470e738640 1 --2- 172.21.15.83:0/3755027335 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4708160690 0x7f470813b910 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.970+0000 7f470e738640 1 --2- 172.21.15.83:0/3755027335 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f47081421e0 0x7f47080d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.970+0000 7f470e738640 1 --2- 172.21.15.83:0/3755027335 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f470806f7b0 0x7f4708163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:06.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.970+0000 7f470e738640 1 -- 172.21.15.83:0/3755027335 >> 172.21.15.83:0/3755027335 conn(0x7f4708068860 msgr2=0x7f470813c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:06.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.970+0000 7f470e738640 1 -- 172.21.15.83:0/3755027335 shutdown_connections 2024-04-26T19:50:06.972 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:06.970+0000 7f470e738640 1 -- 172.21.15.83:0/3755027335 wait complete. 2024-04-26T19:50:06.982 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new e01f7537-cc44-4ed8-9ba3-8f61de55aac1 1 2024-04-26T19:50:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.118+0000 7fd591007640 1 Processor -- start 2024-04-26T19:50:07.119 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.118+0000 7fd591007640 1 -- start start 2024-04-26T19:50:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.118+0000 7fd591007640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd58c1422e0 0x7fd58c160150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.121 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.118+0000 7fd591007640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c06f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.118+0000 7fd591007640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd58c06f5e0 0x7fd58c162fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.118+0000 7fd591007640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd58c144b50 con 0x7fd58c1422e0 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.118+0000 7fd591007640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd58c144e30 con 0x7fd58c06f5e0 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.118+0000 7fd591007640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd58c145110 con 0x7fd58c160690 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58a59c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c06f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58b59e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd58c06f5e0 0x7fd58c162fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58b59e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd58c06f5e0 0x7fd58c162fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44288/0 (socket says 172.21.15.83:44288) 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58a59c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c06f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.83:34642/0 (socket says 172.21.15.83:34642) 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58b59e640 1 -- 172.21.15.83:0/3657577188 learned_addr learned my addr 172.21.15.83:0/3657577188 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58ad9d640 1 --2- 172.21.15.83:0/3657577188 >> v2:172.21.15.79:3300/0 conn(0x7fd58c1422e0 0x7fd58c160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58a59c640 1 -- 172.21.15.83:0/3657577188 >> v2:172.21.15.83:3300/0 conn(0x7fd58c06f5e0 msgr2=0x7fd58c162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58a59c640 1 --2- 172.21.15.83:0/3657577188 >> v2:172.21.15.83:3300/0 conn(0x7fd58c06f5e0 0x7fd58c162fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.122 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58a59c640 1 -- 172.21.15.83:0/3657577188 >> v2:172.21.15.79:3300/0 conn(0x7fd58c1422e0 msgr2=0x7fd58c160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58a59c640 1 --2- 172.21.15.83:0/3657577188 >> v2:172.21.15.79:3300/0 conn(0x7fd58c1422e0 0x7fd58c160150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58a59c640 1 -- 172.21.15.83:0/3657577188 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c1453f0 con 0x7fd58c160690 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.119+0000 7fd58b59e640 1 --2- 172.21.15.83:0/3657577188 >> v2:172.21.15.83:3300/0 conn(0x7fd58c06f5e0 0x7fd58c162fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd58a59c640 1 --2- 172.21.15.83:0/3657577188 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c06f0a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd58c15d510 tx=0x7fd58002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63ea4791445991a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd589d9b640 1 -- 172.21.15.83:0/3657577188 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580037070 con 0x7fd58c160690 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd589d9b640 1 -- 172.21.15.83:0/3657577188 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd580032440 con 0x7fd58c160690 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd589d9b640 1 -- 172.21.15.83:0/3657577188 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58002ae60 con 0x7fd58c160690 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd591007640 1 -- 172.21.15.83:0/3657577188 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 msgr2=0x7fd58c06f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd591007640 1 --2- 172.21.15.83:0/3657577188 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c06f0a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd58c15d510 tx=0x7fd58002a110 comp rx=0 tx=0).stop 2024-04-26T19:50:07.123 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd591007640 1 -- 172.21.15.83:0/3657577188 shutdown_connections 2024-04-26T19:50:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd591007640 1 --2- 172.21.15.83:0/3657577188 >> v2:172.21.15.83:3300/0 conn(0x7fd58c06f5e0 0x7fd58c162fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd591007640 1 --2- 172.21.15.83:0/3657577188 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c06f0a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd591007640 1 --2- 172.21.15.83:0/3657577188 >> v2:172.21.15.79:3300/0 conn(0x7fd58c1422e0 0x7fd58c160150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd591007640 1 -- 172.21.15.83:0/3657577188 >> 172.21.15.83:0/3657577188 conn(0x7fd58c068860 msgr2=0x7fd58c0b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd591007640 1 -- 172.21.15.83:0/3657577188 shutdown_connections 2024-04-26T19:50:07.124 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.120+0000 7fd591007640 1 -- 172.21.15.83:0/3657577188 wait complete. 2024-04-26T19:50:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.122+0000 7fd591007640 1 Processor -- start 2024-04-26T19:50:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.122+0000 7fd591007640 1 -- start start 2024-04-26T19:50:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd591007640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd58c06f5e0 0x7fd58c147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd591007640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd58c1422e0 0x7fd58c148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd591007640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c1584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd591007640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c15fd10 con 0x7fd58c06f5e0 2024-04-26T19:50:07.127 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd591007640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd58c15fb90 con 0x7fd58c1422e0 2024-04-26T19:50:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd591007640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd58c071850 con 0x7fd58c160690 2024-04-26T19:50:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58b59e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c1584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd58c1422e0 0x7fd58c148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd58c1422e0 0x7fd58c148520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44294/0 (socket says 172.21.15.83:44294) 2024-04-26T19:50:07.128 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58a59c640 1 -- 172.21.15.83:0/3322374194 learned_addr learned my addr 172.21.15.83:0/3322374194 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58b59e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c1584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.83:34652/0 (socket says 172.21.15.83:34652) 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58ad9d640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd58c06f5e0 0x7fd58c147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58a59c640 1 -- 172.21.15.83:0/3322374194 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 msgr2=0x7fd58c1584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58a59c640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c1584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58a59c640 1 -- 172.21.15.83:0/3322374194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd58c06f5e0 msgr2=0x7fd58c147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58a59c640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd58c06f5e0 0x7fd58c147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58a59c640 1 -- 172.21.15.83:0/3322374194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58c148a60 con 0x7fd58c1422e0 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58b59e640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c1584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58ad9d640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd58c06f5e0 0x7fd58c147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd58a59c640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd58c1422e0 0x7fd58c148520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd580009890 tx=0x7fd5800046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.123+0000 7fd5737fe640 1 -- 172.21.15.83:0/3322374194 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580041070 con 0x7fd58c1422e0 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.124+0000 7fd5737fe640 1 -- 172.21.15.83:0/3322374194 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58003c890 con 0x7fd58c1422e0 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.124+0000 7fd5737fe640 1 -- 172.21.15.83:0/3322374194 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580037040 con 0x7fd58c1422e0 2024-04-26T19:50:07.129 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.124+0000 7fd591007640 1 -- 172.21.15.83:0/3322374194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58c156eb0 con 0x7fd58c1422e0 2024-04-26T19:50:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.124+0000 7fd591007640 1 -- 172.21.15.83:0/3322374194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58c1438f0 con 0x7fd58c1422e0 2024-04-26T19:50:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.124+0000 7fd5737fe640 1 -- 172.21.15.83:0/3322374194 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd580044070 con 0x7fd58c1422e0 2024-04-26T19:50:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.124+0000 7fd5737fe640 1 -- 172.21.15.83:0/3322374194 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fd580046d90 con 0x7fd58c1422e0 2024-04-26T19:50:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.125+0000 7fd591007640 1 -- 172.21.15.83:0/3322374194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd58c0506c0 con 0x7fd58c1422e0 2024-04-26T19:50:07.130 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.127+0000 7fd5737fe640 1 -- 172.21.15.83:0/3322374194 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fd580046450 con 0x7fd58c1422e0 2024-04-26T19:50:07.200 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.199+0000 7fd591007640 1 -- 172.21.15.83:0/3322374194 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e01f7537-cc44-4ed8-9ba3-8f61de55aac1", "id": 1} v 0) -- 0x7fd58c0afc60 con 0x7fd58c1422e0 2024-04-26T19:50:07.205 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.204+0000 7fd5737fe640 1 -- 172.21.15.83:0/3322374194 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e01f7537-cc44-4ed8-9ba3-8f61de55aac1", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fd58003cc80 con 0x7fd58c1422e0 2024-04-26T19:50:07.206 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-26T19:50:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.205+0000 7fd591007640 1 -- 172.21.15.83:0/3322374194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd58c1422e0 msgr2=0x7fd58c148520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.205+0000 7fd591007640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd58c1422e0 0x7fd58c148520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd580009890 tx=0x7fd5800046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.205+0000 7fd591007640 1 -- 172.21.15.83:0/3322374194 shutdown_connections 2024-04-26T19:50:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.205+0000 7fd591007640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd58c160690 0x7fd58c1584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.205+0000 7fd591007640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd58c1422e0 0x7fd58c148520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.205+0000 7fd591007640 1 --2- 172.21.15.83:0/3322374194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd58c06f5e0 0x7fd58c147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.205+0000 7fd591007640 1 -- 172.21.15.83:0/3322374194 >> 172.21.15.83:0/3322374194 conn(0x7fd58c068860 msgr2=0x7fd58c072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.205+0000 7fd591007640 1 -- 172.21.15.83:0/3322374194 shutdown_connections 2024-04-26T19:50:07.207 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.205+0000 7fd591007640 1 -- 172.21.15.83:0/3322374194 wait complete. 2024-04-26T19:50:07.219 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 428e11d4-edb8-4003-a6b6-59141d177026 2 2024-04-26T19:50:07.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.315+0000 7fab3a1b4640 1 Processor -- start 2024-04-26T19:50:07.317 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.315+0000 7fab3a1b4640 1 -- start start 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.316+0000 7fab3a1b4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab340b69c0 0x7fab3406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.316+0000 7fab3a1b4640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fab3406f380 0x7fab34072bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.316+0000 7fab3a1b4640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab3406fb70 con 0x7fab3415cf00 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.316+0000 7fab3a1b4640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fab34160b00 con 0x7fab3406f380 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.316+0000 7fab3a1b4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fab3406fe50 con 0x7fab340b69c0 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.316+0000 7fab2affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab340b69c0 0x7fab3406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.316+0000 7fab2affd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab340b69c0 0x7fab3406ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44302/0 (socket says 172.21.15.83:44302) 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.317+0000 7fab2affd640 1 -- 172.21.15.83:0/3864841298 learned_addr learned my addr 172.21.15.83:0/3864841298 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.317+0000 7fab33fff640 1 --2- 172.21.15.83:0/3864841298 >> v2:172.21.15.79:3300/0 conn(0x7fab3406f380 0x7fab34072bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.317+0000 7fab2affd640 1 -- 172.21.15.83:0/3864841298 >> v2:172.21.15.79:3300/0 conn(0x7fab3406f380 msgr2=0x7fab34072bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:07.320 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.317+0000 7fab2affd640 1 --2- 172.21.15.83:0/3864841298 >> v2:172.21.15.79:3300/0 conn(0x7fab3406f380 0x7fab34072bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.317+0000 7fab2affd640 1 -- 172.21.15.83:0/3864841298 >> v1:172.21.15.79:6789/0 conn(0x7fab3415cf00 legacy=0x7fab340b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.317+0000 7fab2affd640 1 -- 172.21.15.83:0/3864841298 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab34070130 con 0x7fab340b69c0 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.317+0000 7fab33fff640 1 --2- 172.21.15.83:0/3864841298 >> v2:172.21.15.79:3300/0 conn(0x7fab3406f380 0x7fab34072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.317+0000 7fab2affd640 1 --2- 172.21.15.83:0/3864841298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab340b69c0 0x7fab3406ee40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fab340b3b60 tx=0x7fab1c02a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cc267282afc9ee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab32ffd640 1 -- 172.21.15.83:0/3864841298 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c037070 con 0x7fab340b69c0 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab32ffd640 1 -- 172.21.15.83:0/3864841298 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab1c032440 con 0x7fab340b69c0 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab32ffd640 1 -- 172.21.15.83:0/3864841298 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c032810 con 0x7fab340b69c0 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab3a1b4640 1 -- 172.21.15.83:0/3864841298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab340b69c0 msgr2=0x7fab3406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab3a1b4640 1 --2- 172.21.15.83:0/3864841298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab340b69c0 0x7fab3406ee40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fab340b3b60 tx=0x7fab1c02a270 comp rx=0 tx=0).stop 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab3a1b4640 1 -- 172.21.15.83:0/3864841298 shutdown_connections 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab3a1b4640 1 --2- 172.21.15.83:0/3864841298 >> v2:172.21.15.79:3300/0 conn(0x7fab3406f380 0x7fab34072bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab3a1b4640 1 --2- 172.21.15.83:0/3864841298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab340b69c0 0x7fab3406ee40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab3a1b4640 1 -- 172.21.15.83:0/3864841298 >> 172.21.15.83:0/3864841298 conn(0x7fab3406cdd0 msgr2=0x7fab340b2f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab3a1b4640 1 -- 172.21.15.83:0/3864841298 shutdown_connections 2024-04-26T19:50:07.321 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.318+0000 7fab3a1b4640 1 -- 172.21.15.83:0/3864841298 wait complete. 2024-04-26T19:50:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.319+0000 7fab3a1b4640 1 Processor -- start 2024-04-26T19:50:07.325 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab3a1b4640 1 -- start start 2024-04-26T19:50:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab3a1b4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3406f380 0x7fab34145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab3a1b4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab340b69c0 0x7fab341f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab3a1b4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3415cf00 0x7fab34158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab3a1b4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fab340722e0 con 0x7fab3406f380 2024-04-26T19:50:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab3a1b4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab3415d2e0 con 0x7fab3415cf00 2024-04-26T19:50:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab3a1b4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fab34072460 con 0x7fab340b69c0 2024-04-26T19:50:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab33fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3415cf00 0x7fab34158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab2affd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab340b69c0 0x7fab341f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.326 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab33fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3415cf00 0x7fab34158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44304/0 (socket says 172.21.15.83:44304) 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab2affd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab340b69c0 0x7fab341f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.83:34662/0 (socket says 172.21.15.83:34662) 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab33fff640 1 -- 172.21.15.83:0/4072277299 learned_addr learned my addr 172.21.15.83:0/4072277299 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.320+0000 7fab337fe640 1 --2- 172.21.15.83:0/4072277299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3406f380 0x7fab34145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab2affd640 1 -- 172.21.15.83:0/4072277299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3415cf00 msgr2=0x7fab34158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab2affd640 1 --2- 172.21.15.83:0/4072277299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3415cf00 0x7fab34158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab2affd640 1 -- 172.21.15.83:0/4072277299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3406f380 msgr2=0x7fab34145c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab2affd640 1 --2- 172.21.15.83:0/4072277299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3406f380 0x7fab34145c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab2affd640 1 -- 172.21.15.83:0/4072277299 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab341f57c0 con 0x7fab340b69c0 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab33fff640 1 --2- 172.21.15.83:0/4072277299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3415cf00 0x7fab34158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab2affd640 1 --2- 172.21.15.83:0/4072277299 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab340b69c0 0x7fab341f5280 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fab34160c80 tx=0x7fab1c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab30ff9640 1 -- 172.21.15.83:0/4072277299 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c041020 con 0x7fab340b69c0 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab30ff9640 1 -- 172.21.15.83:0/4072277299 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab1c030700 con 0x7fab340b69c0 2024-04-26T19:50:07.327 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab3a1b4640 1 -- 172.21.15.83:0/4072277299 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab341598b0 con 0x7fab340b69c0 2024-04-26T19:50:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab30ff9640 1 -- 172.21.15.83:0/4072277299 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c037040 con 0x7fab340b69c0 2024-04-26T19:50:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab3a1b4640 1 -- 172.21.15.83:0/4072277299 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fab34145190 con 0x7fab340b69c0 2024-04-26T19:50:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab30ff9640 1 -- 172.21.15.83:0/4072277299 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab1c044020 con 0x7fab340b69c0 2024-04-26T19:50:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.321+0000 7fab30ff9640 1 -- 172.21.15.83:0/4072277299 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fab1c03dc40 con 0x7fab340b69c0 2024-04-26T19:50:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.322+0000 7fab3a1b4640 1 -- 172.21.15.83:0/4072277299 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab341457a0 con 0x7fab340b69c0 2024-04-26T19:50:07.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.324+0000 7fab30ff9640 1 -- 172.21.15.83:0/4072277299 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fab1c04b030 con 0x7fab340b69c0 2024-04-26T19:50:07.397 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.396+0000 7fab3a1b4640 1 -- 172.21.15.83:0/4072277299 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd new", "uuid": "428e11d4-edb8-4003-a6b6-59141d177026", "id": 2} v 0) -- 0x7fab340506c0 con 0x7fab340b69c0 2024-04-26T19:50:07.402 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.401+0000 7fab30ff9640 1 -- 172.21.15.83:0/4072277299 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "428e11d4-edb8-4003-a6b6-59141d177026", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fab1c03d420 con 0x7fab340b69c0 2024-04-26T19:50:07.402 INFO:teuthology.orchestra.run.smithi083.stdout:2 2024-04-26T19:50:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.402+0000 7fab3a1b4640 1 -- 172.21.15.83:0/4072277299 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab340b69c0 msgr2=0x7fab341f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.403 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.402+0000 7fab3a1b4640 1 --2- 172.21.15.83:0/4072277299 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab340b69c0 0x7fab341f5280 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fab34160c80 tx=0x7fab1c0046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.402+0000 7fab3a1b4640 1 -- 172.21.15.83:0/4072277299 shutdown_connections 2024-04-26T19:50:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.402+0000 7fab3a1b4640 1 --2- 172.21.15.83:0/4072277299 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab3415cf00 0x7fab34158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.402+0000 7fab3a1b4640 1 --2- 172.21.15.83:0/4072277299 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab340b69c0 0x7fab341f5280 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.402+0000 7fab3a1b4640 1 --2- 172.21.15.83:0/4072277299 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab3406f380 0x7fab34145c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.402+0000 7fab3a1b4640 1 -- 172.21.15.83:0/4072277299 >> 172.21.15.83:0/4072277299 conn(0x7fab3406cdd0 msgr2=0x7fab34160370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.402+0000 7fab3a1b4640 1 -- 172.21.15.83:0/4072277299 shutdown_connections 2024-04-26T19:50:07.404 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.402+0000 7fab3a1b4640 1 -- 172.21.15.83:0/4072277299 wait complete. 2024-04-26T19:50:07.414 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 89c30ecc-bd6d-43a7-829e-57e01ad3179e 3 2024-04-26T19:50:07.514 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.510+0000 7fc1336bd640 1 Processor -- start 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.510+0000 7fc1336bd640 1 -- start start 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc1336bd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c15e280 0x7fc12c0d10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc1336bd640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12c143880 con 0x7fc12c1434a0 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc1336bd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc12c15d7c0 con 0x7fc12c0d1600 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc1336bd640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c161830 con 0x7fc12c15e280 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc130c58640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c15e280 0x7fc12c0d10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc131c5a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc12c0d1600 0x7fc12c1408d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51454/0 (socket says 172.21.15.83:51454) 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc131c5a640 1 -- 172.21.15.83:0/2056045386 learned_addr learned my addr 172.21.15.83:0/2056045386 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc123fff640 1 -- 172.21.15.83:0/2056045386 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2950817426 0 0) 0x7fc12c15d7c0 con 0x7fc12c0d1600 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc130c58640 1 -- 172.21.15.83:0/2056045386 >> v1:172.21.15.83:6789/0 conn(0x7fc12c0d1600 legacy=0x7fc12c1408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc130c58640 1 -- 172.21.15.83:0/2056045386 >> v1:172.21.15.79:6789/0 conn(0x7fc12c1434a0 legacy=0x7fc12c160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.511+0000 7fc130c58640 1 -- 172.21.15.83:0/2056045386 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c161b10 con 0x7fc12c15e280 2024-04-26T19:50:07.515 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.512+0000 7fc130c58640 1 --2- 172.21.15.83:0/2056045386 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c15e280 0x7fc12c0d10c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0097d0 tx=0x7fc11c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=781844faa25224fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.512+0000 7fc123fff640 1 -- 172.21.15.83:0/2056045386 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c037070 con 0x7fc12c15e280 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.512+0000 7fc123fff640 1 -- 172.21.15.83:0/2056045386 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc11c032440 con 0x7fc12c15e280 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.512+0000 7fc123fff640 1 -- 172.21.15.83:0/2056045386 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c032810 con 0x7fc12c15e280 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.512+0000 7fc1336bd640 1 -- 172.21.15.83:0/2056045386 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c15e280 msgr2=0x7fc12c0d10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.512+0000 7fc1336bd640 1 --2- 172.21.15.83:0/2056045386 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c15e280 0x7fc12c0d10c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc11c0097d0 tx=0x7fc11c02a180 comp rx=0 tx=0).stop 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.512+0000 7fc1336bd640 1 -- 172.21.15.83:0/2056045386 shutdown_connections 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.512+0000 7fc1336bd640 1 --2- 172.21.15.83:0/2056045386 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c15e280 0x7fc12c0d10c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.512+0000 7fc1336bd640 1 -- 172.21.15.83:0/2056045386 >> 172.21.15.83:0/2056045386 conn(0x7fc12c068860 msgr2=0x7fc12c0b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.513+0000 7fc1336bd640 1 -- 172.21.15.83:0/2056045386 shutdown_connections 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.513+0000 7fc1336bd640 1 -- 172.21.15.83:0/2056045386 wait complete. 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc1336bd640 1 Processor -- start 2024-04-26T19:50:07.516 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc1336bd640 1 -- start start 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc1336bd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c0d1600 0x7fc12c159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc1336bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12c1434a0 0x7fc12c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc1336bd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc12c15e280 0x7fc12c0ac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc1336bd640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c15e700 con 0x7fc12c15e280 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc1336bd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc12c15dd60 con 0x7fc12c1434a0 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc1336bd640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc12c1614c0 con 0x7fc12c0d1600 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc131459640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c0d1600 0x7fc12c159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc130c58640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12c1434a0 0x7fc12c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc131459640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c0d1600 0x7fc12c159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.83:34674/0 (socket says 172.21.15.83:34674) 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.514+0000 7fc131459640 1 -- 172.21.15.83:0/3919009096 learned_addr learned my addr 172.21.15.83:0/3919009096 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc131c5a640 1 --2- 172.21.15.83:0/3919009096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc12c15e280 0x7fc12c0ac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc130c58640 1 -- 172.21.15.83:0/3919009096 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c0d1600 msgr2=0x7fc12c159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.519 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc130c58640 1 --2- 172.21.15.83:0/3919009096 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c0d1600 0x7fc12c159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc130c58640 1 -- 172.21.15.83:0/3919009096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc12c15e280 msgr2=0x7fc12c0ac000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc130c58640 1 --2- 172.21.15.83:0/3919009096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc12c15e280 0x7fc12c0ac000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc130c58640 1 -- 172.21.15.83:0/3919009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc12c1584e0 con 0x7fc12c1434a0 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc131459640 1 --2- 172.21.15.83:0/3919009096 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c0d1600 0x7fc12c159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc130c58640 1 --2- 172.21.15.83:0/3919009096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12c1434a0 0x7fc12c15a0f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc11c031000 tx=0x7fc11c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc121ffb640 1 -- 172.21.15.83:0/3919009096 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c042020 con 0x7fc12c1434a0 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc121ffb640 1 -- 172.21.15.83:0/3919009096 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc11c038040 con 0x7fc12c1434a0 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc1336bd640 1 -- 172.21.15.83:0/3919009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc12c1587c0 con 0x7fc12c1434a0 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc121ffb640 1 -- 172.21.15.83:0/3919009096 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc11c037050 con 0x7fc12c1434a0 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc1336bd640 1 -- 172.21.15.83:0/3919009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc12c158d80 con 0x7fc12c1434a0 2024-04-26T19:50:07.520 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc121ffb640 1 -- 172.21.15.83:0/3919009096 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc11c030900 con 0x7fc12c1434a0 2024-04-26T19:50:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.515+0000 7fc121ffb640 1 -- 172.21.15.83:0/3919009096 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fc11c003dc0 con 0x7fc12c1434a0 2024-04-26T19:50:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.516+0000 7fc1336bd640 1 -- 172.21.15.83:0/3919009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc12c0506c0 con 0x7fc12c1434a0 2024-04-26T19:50:07.521 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.518+0000 7fc121ffb640 1 -- 172.21.15.83:0/3919009096 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc11c04c020 con 0x7fc12c1434a0 2024-04-26T19:50:07.535 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:07.535+0000 7fb7d1810040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:50:07.593 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.591+0000 7fc1336bd640 1 -- 172.21.15.83:0/3919009096 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "89c30ecc-bd6d-43a7-829e-57e01ad3179e", "id": 3} v 0) -- 0x7fc12c073e00 con 0x7fc12c1434a0 2024-04-26T19:50:07.595 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.594+0000 7fc121ffb640 1 -- 172.21.15.83:0/3919009096 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "89c30ecc-bd6d-43a7-829e-57e01ad3179e", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fc11c0044a0 con 0x7fc12c1434a0 2024-04-26T19:50:07.596 INFO:teuthology.orchestra.run.smithi083.stdout:3 2024-04-26T19:50:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.595+0000 7fc1336bd640 1 -- 172.21.15.83:0/3919009096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12c1434a0 msgr2=0x7fc12c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.595+0000 7fc1336bd640 1 --2- 172.21.15.83:0/3919009096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12c1434a0 0x7fc12c15a0f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc11c031000 tx=0x7fc11c0046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.595+0000 7fc1336bd640 1 -- 172.21.15.83:0/3919009096 shutdown_connections 2024-04-26T19:50:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.595+0000 7fc1336bd640 1 --2- 172.21.15.83:0/3919009096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc12c15e280 0x7fc12c0ac000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.595+0000 7fc1336bd640 1 --2- 172.21.15.83:0/3919009096 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc12c1434a0 0x7fc12c15a0f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.595+0000 7fc1336bd640 1 --2- 172.21.15.83:0/3919009096 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc12c0d1600 0x7fc12c159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.595+0000 7fc1336bd640 1 -- 172.21.15.83:0/3919009096 >> 172.21.15.83:0/3919009096 conn(0x7fc12c068860 msgr2=0x7fc12c1429d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:07.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.596+0000 7fc1336bd640 1 -- 172.21.15.83:0/3919009096 shutdown_connections 2024-04-26T19:50:07.598 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.596+0000 7fc1336bd640 1 -- 172.21.15.83:0/3919009096 wait complete. 2024-04-26T19:50:07.607 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new a88b8475-38de-4d46-bf92-12cb08f29cd6 4 2024-04-26T19:50:07.616 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:07.615+0000 7fdb97d21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:50:07.620 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:07.620+0000 7fb7d1810040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:50:07.662 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:07.662+0000 7fb7d1810040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:50:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.703+0000 7f878ba25640 1 Processor -- start 2024-04-26T19:50:07.704 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.703+0000 7f878ba25640 1 -- start start 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f878ba25640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f87840cf1b0 0x7f8784073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f878ba25640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87840b0ae0 con 0x7f87840b1b80 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f878ba25640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87840b0dc0 con 0x7f878415f310 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f878ba25640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f87840b16f0 con 0x7f87840cf1b0 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f87897c1640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f878415f310 0x7f87840ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51460/0 (socket says 172.21.15.83:51460) 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f87897c1640 1 -- 172.21.15.83:0/923060031 learned_addr learned my addr 172.21.15.83:0/923060031 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f8788fc0640 1 --2- 172.21.15.83:0/923060031 >> v2:172.21.15.79:3300/0 conn(0x7f87840cf1b0 0x7f8784073ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f877bfff640 1 -- 172.21.15.83:0/923060031 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 918965754 0 0) 0x7f87840b0dc0 con 0x7f878415f310 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f877bfff640 1 -- 172.21.15.83:0/923060031 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8770003520 con 0x7f878415f310 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f877bfff640 1 -- 172.21.15.83:0/923060031 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2628625477 0 0) 0x7f87840b0ae0 con 0x7f87840b1b80 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.704+0000 7f877bfff640 1 -- 172.21.15.83:0/923060031 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87840b0dc0 con 0x7f87840b1b80 2024-04-26T19:50:07.706 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.705+0000 7f8788fc0640 1 -- 172.21.15.83:0/923060031 >> v1:172.21.15.83:6789/0 conn(0x7f878415f310 legacy=0x7f87840ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.705+0000 7f8788fc0640 1 -- 172.21.15.83:0/923060031 >> v1:172.21.15.79:6789/0 conn(0x7f87840b1b80 legacy=0x7f87840aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.707 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.705+0000 7f8788fc0640 1 -- 172.21.15.83:0/923060031 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87840b10a0 con 0x7f87840cf1b0 2024-04-26T19:50:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.705+0000 7f8788fc0640 1 --2- 172.21.15.83:0/923060031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87840cf1b0 0x7f8784073ec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f87800097a0 tx=0x7f878002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15bb82b43c167d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.705+0000 7f877bfff640 1 -- 172.21.15.83:0/923060031 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780037070 con 0x7f87840cf1b0 2024-04-26T19:50:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.705+0000 7f877bfff640 1 -- 172.21.15.83:0/923060031 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8780032440 con 0x7f87840cf1b0 2024-04-26T19:50:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.705+0000 7f877bfff640 1 -- 172.21.15.83:0/923060031 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780032810 con 0x7f87840cf1b0 2024-04-26T19:50:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.706+0000 7f878ba25640 1 -- 172.21.15.83:0/923060031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87840cf1b0 msgr2=0x7f8784073ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.706+0000 7f878ba25640 1 --2- 172.21.15.83:0/923060031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87840cf1b0 0x7f8784073ec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f87800097a0 tx=0x7f878002a230 comp rx=0 tx=0).stop 2024-04-26T19:50:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.706+0000 7f878ba25640 1 -- 172.21.15.83:0/923060031 shutdown_connections 2024-04-26T19:50:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.706+0000 7f878ba25640 1 --2- 172.21.15.83:0/923060031 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87840cf1b0 0x7f8784073ec0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.708 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.706+0000 7f878ba25640 1 -- 172.21.15.83:0/923060031 >> 172.21.15.83:0/923060031 conn(0x7f87840b3ed0 msgr2=0x7f8784068180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:07.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.706+0000 7f878ba25640 1 -- 172.21.15.83:0/923060031 shutdown_connections 2024-04-26T19:50:07.709 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.706+0000 7f878ba25640 1 -- 172.21.15.83:0/923060031 wait complete. 2024-04-26T19:50:07.709 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:07.707+0000 7fdb97d21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:50:07.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.707+0000 7f878ba25640 1 Processor -- start 2024-04-26T19:50:07.711 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.707+0000 7f878ba25640 1 -- start start 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f878ba25640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87840b1b80 0x7f8784161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f878ba25640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87840cf1b0 0x7f8784156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f878ba25640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f878415f310 0x7f8784153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f878ba25640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f878415f870 con 0x7f87840b1b80 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f878ba25640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f878415f6f0 con 0x7f87840cf1b0 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f878ba25640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f878415f9f0 con 0x7f878415f310 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8788fc0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87840cf1b0 0x7f8784156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8789fc2640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f878415f310 0x7f8784153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8788fc0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87840cf1b0 0x7f8784156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44334/0 (socket says 172.21.15.83:44334) 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8788fc0640 1 -- 172.21.15.83:0/3426268126 learned_addr learned my addr 172.21.15.83:0/3426268126 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f87897c1640 1 --2- 172.21.15.83:0/3426268126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87840b1b80 0x7f8784161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8789fc2640 1 -- 172.21.15.83:0/3426268126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87840cf1b0 msgr2=0x7f8784156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8789fc2640 1 --2- 172.21.15.83:0/3426268126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87840cf1b0 0x7f8784156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8789fc2640 1 -- 172.21.15.83:0/3426268126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87840b1b80 msgr2=0x7f8784161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.712 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8789fc2640 1 --2- 172.21.15.83:0/3426268126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87840b1b80 0x7f8784161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8789fc2640 1 -- 172.21.15.83:0/3426268126 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8784153790 con 0x7f878415f310 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.708+0000 7f8788fc0640 1 --2- 172.21.15.83:0/3426268126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87840cf1b0 0x7f8784156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.709+0000 7f8789fc2640 1 --2- 172.21.15.83:0/3426268126 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f878415f310 0x7f8784153250 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f877c00c6c0 tx=0x7f877c00cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.709+0000 7f8779ffb640 1 -- 172.21.15.83:0/3426268126 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c003b70 con 0x7f878415f310 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.709+0000 7f8779ffb640 1 -- 172.21.15.83:0/3426268126 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f877c003d10 con 0x7f878415f310 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.709+0000 7f878ba25640 1 -- 172.21.15.83:0/3426268126 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8784154200 con 0x7f878415f310 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.709+0000 7f8779ffb640 1 -- 172.21.15.83:0/3426268126 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c005310 con 0x7f878415f310 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.709+0000 7f8779ffb640 1 -- 172.21.15.83:0/3426268126 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f877c0054b0 con 0x7f878415f310 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.709+0000 7f878ba25640 1 -- 172.21.15.83:0/3426268126 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f87841627b0 con 0x7f878415f310 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.709+0000 7f8779ffb640 1 -- 172.21.15.83:0/3426268126 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f877c00ab50 con 0x7f878415f310 2024-04-26T19:50:07.713 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.710+0000 7f878ba25640 1 -- 172.21.15.83:0/3426268126 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87840cfa80 con 0x7f878415f310 2024-04-26T19:50:07.714 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.712+0000 7f8779ffb640 1 -- 172.21.15.83:0/3426268126 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f877c020020 con 0x7f878415f310 2024-04-26T19:50:07.752 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:07.751+0000 7fdb97d21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:50:07.785 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.784+0000 7f878ba25640 1 -- 172.21.15.83:0/3426268126 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a88b8475-38de-4d46-bf92-12cb08f29cd6", "id": 4} v 0) -- 0x7f878415a610 con 0x7f878415f310 2024-04-26T19:50:07.790 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.789+0000 7f8779ffb640 1 -- 172.21.15.83:0/3426268126 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a88b8475-38de-4d46-bf92-12cb08f29cd6", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f8784153790 con 0x7f878415f310 2024-04-26T19:50:07.791 INFO:teuthology.orchestra.run.smithi083.stdout:4 2024-04-26T19:50:07.791 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.790+0000 7f878ba25640 1 -- 172.21.15.83:0/3426268126 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f878415f310 msgr2=0x7f8784153250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.790+0000 7f878ba25640 1 --2- 172.21.15.83:0/3426268126 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f878415f310 0x7f8784153250 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f877c00c6c0 tx=0x7f877c00cb80 comp rx=0 tx=0).stop 2024-04-26T19:50:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.790+0000 7f878ba25640 1 -- 172.21.15.83:0/3426268126 shutdown_connections 2024-04-26T19:50:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.790+0000 7f878ba25640 1 --2- 172.21.15.83:0/3426268126 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f878415f310 0x7f8784153250 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.790+0000 7f878ba25640 1 --2- 172.21.15.83:0/3426268126 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f87840cf1b0 0x7f8784156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.790+0000 7f878ba25640 1 --2- 172.21.15.83:0/3426268126 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f87840b1b80 0x7f8784161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.790+0000 7f878ba25640 1 -- 172.21.15.83:0/3426268126 >> 172.21.15.83:0/3426268126 conn(0x7f87840b3ed0 msgr2=0x7f87840d19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.790+0000 7f878ba25640 1 -- 172.21.15.83:0/3426268126 shutdown_connections 2024-04-26T19:50:07.792 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.790+0000 7f878ba25640 1 -- 172.21.15.83:0/3426268126 wait complete. 2024-04-26T19:50:07.802 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 0334aebf-d29f-4622-bd30-c69630e9c7ae 5 2024-04-26T19:50:07.825 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:07.824+0000 7fb7d1810040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:50:07.875 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:07.874+0000 7fb7d1810040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:50:07.926 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:07.924+0000 7fdb97d21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:50:08.042 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:07.925+0000 7fb7d1810040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:50:08.042 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:08.036+0000 7fb7d1810040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:50:08.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.938+0000 7f8b3af11640 1 Processor -- start 2024-04-26T19:50:08.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.938+0000 7f8b3af11640 1 -- start start 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b3af11640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8b3406f7b0 0x7f8b34141800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b3af11640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b34073570 0x7f8b34141d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b3af11640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b34142410 con 0x7f8b3413eff0 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b3af11640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f8b341426f0 con 0x7f8b3406f7b0 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b3af11640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8b34074770 con 0x7f8b34073570 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b394ae640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b34073570 0x7f8b34141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b33fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8b3406f7b0 0x7f8b34141800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b394ae640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8b34073570 0x7f8b34141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44348/0 (socket says 172.21.15.83:44348) 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b394ae640 1 -- 172.21.15.83:0/846673389 learned_addr learned my addr 172.21.15.83:0/846673389 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b394ae640 1 -- 172.21.15.83:0/846673389 >> v2:172.21.15.79:3300/0 conn(0x7f8b3406f7b0 msgr2=0x7f8b34141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b394ae640 1 --2- 172.21.15.83:0/846673389 >> v2:172.21.15.79:3300/0 conn(0x7f8b3406f7b0 0x7f8b34141800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b394ae640 1 -- 172.21.15.83:0/846673389 >> v1:172.21.15.79:6789/0 conn(0x7f8b3413eff0 legacy=0x7f8b3406f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.939+0000 7f8b394ae640 1 -- 172.21.15.83:0/846673389 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b34072a90 con 0x7f8b34073570 2024-04-26T19:50:08.047 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.940+0000 7f8b394ae640 1 --2- 172.21.15.83:0/846673389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b34073570 0x7f8b34141d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b340b1620 tx=0x7f8b2c02a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf9256af42df3a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.940+0000 7f8b337fe640 1 -- 172.21.15.83:0/846673389 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c036070 con 0x7f8b34073570 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.940+0000 7f8b337fe640 1 -- 172.21.15.83:0/846673389 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b2c031600 con 0x7f8b34073570 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.940+0000 7f8b337fe640 1 -- 172.21.15.83:0/846673389 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c0319d0 con 0x7f8b34073570 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.941+0000 7f8b3af11640 1 -- 172.21.15.83:0/846673389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b34073570 msgr2=0x7f8b34141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.941+0000 7f8b3af11640 1 --2- 172.21.15.83:0/846673389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b34073570 0x7f8b34141d40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8b340b1620 tx=0x7f8b2c02a6a0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.941+0000 7f8b3af11640 1 -- 172.21.15.83:0/846673389 shutdown_connections 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.941+0000 7f8b3af11640 1 --2- 172.21.15.83:0/846673389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b34073570 0x7f8b34141d40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.941+0000 7f8b3af11640 1 --2- 172.21.15.83:0/846673389 >> v2:172.21.15.79:3300/0 conn(0x7f8b3406f7b0 0x7f8b34141800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.941+0000 7f8b3af11640 1 -- 172.21.15.83:0/846673389 >> 172.21.15.83:0/846673389 conn(0x7f8b34068860 msgr2=0x7f8b340b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.941+0000 7f8b3af11640 1 -- 172.21.15.83:0/846673389 shutdown_connections 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.941+0000 7f8b3af11640 1 -- 172.21.15.83:0/846673389 wait complete. 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.942+0000 7f8b3af11640 1 Processor -- start 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.942+0000 7f8b3af11640 1 -- start start 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b3af11640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8b3406f7b0 0x7f8b341f2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.048 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b3af11640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b34073570 0x7f8b340cf990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b3af11640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3413eff0 0x7f8b340cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b3af11640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3413f3d0 con 0x7f8b34073570 2024-04-26T19:50:08.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b3af11640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8b34073050 con 0x7f8b3413eff0 2024-04-26T19:50:08.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b3af11640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8b3413f770 con 0x7f8b3406f7b0 2024-04-26T19:50:08.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b394ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3413eff0 0x7f8b340cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b38cad640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8b3406f7b0 0x7f8b341f2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b394ae640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3413eff0 0x7f8b340cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44354/0 (socket says 172.21.15.83:44354) 2024-04-26T19:50:08.049 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b394ae640 1 -- 172.21.15.83:0/763803615 learned_addr learned my addr 172.21.15.83:0/763803615 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:08.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.943+0000 7f8b33fff640 1 --2- 172.21.15.83:0/763803615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b34073570 0x7f8b340cf990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b38cad640 1 -- 172.21.15.83:0/763803615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3413eff0 msgr2=0x7f8b340cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b38cad640 1 --2- 172.21.15.83:0/763803615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3413eff0 0x7f8b340cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b38cad640 1 -- 172.21.15.83:0/763803615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b34073570 msgr2=0x7f8b340cf990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b38cad640 1 --2- 172.21.15.83:0/763803615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b34073570 0x7f8b340cf990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b38cad640 1 -- 172.21.15.83:0/763803615 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b341f3360 con 0x7f8b3406f7b0 2024-04-26T19:50:08.051 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b394ae640 1 --2- 172.21.15.83:0/763803615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3413eff0 0x7f8b340cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b38cad640 1 --2- 172.21.15.83:0/763803615 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8b3406f7b0 0x7f8b341f2e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b34140780 tx=0x7f8b2800c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b317fa640 1 -- 172.21.15.83:0/763803615 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28017070 con 0x7f8b3406f7b0 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b317fa640 1 -- 172.21.15.83:0/763803615 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b280052d0 con 0x7f8b3406f7b0 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b317fa640 1 -- 172.21.15.83:0/763803615 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b28003850 con 0x7f8b3406f7b0 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b3af11640 1 -- 172.21.15.83:0/763803615 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3412a3d0 con 0x7f8b3406f7b0 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b3af11640 1 -- 172.21.15.83:0/763803615 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b3412a990 con 0x7f8b3406f7b0 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b317fa640 1 -- 172.21.15.83:0/763803615 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8b2801a070 con 0x7f8b3406f7b0 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.944+0000 7f8b317fa640 1 -- 172.21.15.83:0/763803615 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f8b28011590 con 0x7f8b3406f7b0 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.945+0000 7f8b3af11640 1 -- 172.21.15.83:0/763803615 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b340506c0 con 0x7f8b3406f7b0 2024-04-26T19:50:08.052 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:07.947+0000 7f8b317fa640 1 -- 172.21.15.83:0/763803615 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8b28021020 con 0x7f8b3406f7b0 2024-04-26T19:50:08.053 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:07.986+0000 7fdb97d21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:50:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.019+0000 7f8b3af11640 1 -- 172.21.15.83:0/763803615 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0334aebf-d29f-4622-bd30-c69630e9c7ae", "id": 5} v 0) -- 0x7f8b341532d0 con 0x7f8b3406f7b0 2024-04-26T19:50:08.053 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.025+0000 7f8b317fa640 1 -- 172.21.15.83:0/763803615 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0334aebf-d29f-4622-bd30-c69630e9c7ae", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f8b28011b30 con 0x7f8b3406f7b0 2024-04-26T19:50:08.053 INFO:teuthology.orchestra.run.smithi083.stdout:5 2024-04-26T19:50:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.026+0000 7f8b3af11640 1 -- 172.21.15.83:0/763803615 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8b3406f7b0 msgr2=0x7f8b341f2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.026+0000 7f8b3af11640 1 --2- 172.21.15.83:0/763803615 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8b3406f7b0 0x7f8b341f2e20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8b34140780 tx=0x7f8b2800c820 comp rx=0 tx=0).stop 2024-04-26T19:50:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.026+0000 7f8b3af11640 1 -- 172.21.15.83:0/763803615 shutdown_connections 2024-04-26T19:50:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.026+0000 7f8b3af11640 1 --2- 172.21.15.83:0/763803615 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8b3413eff0 0x7f8b340cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.026+0000 7f8b3af11640 1 --2- 172.21.15.83:0/763803615 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8b34073570 0x7f8b340cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.026+0000 7f8b3af11640 1 --2- 172.21.15.83:0/763803615 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8b3406f7b0 0x7f8b341f2e20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.026+0000 7f8b3af11640 1 -- 172.21.15.83:0/763803615 >> 172.21.15.83:0/763803615 conn(0x7f8b34068860 msgr2=0x7f8b340d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.026+0000 7f8b3af11640 1 -- 172.21.15.83:0/763803615 shutdown_connections 2024-04-26T19:50:08.056 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.026+0000 7f8b3af11640 1 -- 172.21.15.83:0/763803615 wait complete. 2024-04-26T19:50:08.057 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new baf0c717-e2cd-4c39-aea1-5b69fab094f3 6 2024-04-26T19:50:08.057 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:08.041+0000 7fdb97d21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:50:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.152+0000 7fab62146640 1 Processor -- start 2024-04-26T19:50:08.154 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.152+0000 7fab62146640 1 -- start start 2024-04-26T19:50:08.162 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.153+0000 7fab62146640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab5c15b810 0x7fab5c15f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.153+0000 7fab62146640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fab5c06feb0 0x7fab5c071af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.153+0000 7fab62146640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fab5c0cee20 con 0x7fab5c0d1310 2024-04-26T19:50:08.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.153+0000 7fab62146640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fab5c0ce210 con 0x7fab5c06feb0 2024-04-26T19:50:08.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.153+0000 7fab62146640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fab5c0ce4f0 con 0x7fab5c15b810 2024-04-26T19:50:08.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.153+0000 7fab5b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fab5c15b810 0x7fab5c15f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.163 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.153+0000 7fab53fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fab5c0d1310 0x7fab5c06f7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.83:51472/0 (socket says 172.21.15.83:51472) 2024-04-26T19:50:08.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.153+0000 7fab53fff640 1 -- 172.21.15.83:0/3318712718 learned_addr learned my addr 172.21.15.83:0/3318712718 (peer_addr_for_me v1:172.21.15.83:0/0) 2024-04-26T19:50:08.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.153+0000 7fab5bfff640 1 --2- 172.21.15.83:0/3318712718 >> v2:172.21.15.79:3300/0 conn(0x7fab5c06feb0 0x7fab5c071af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.154+0000 7fab5b7fe640 1 -- 172.21.15.83:0/3318712718 >> v2:172.21.15.79:3300/0 conn(0x7fab5c06feb0 msgr2=0x7fab5c071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:08.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.154+0000 7fab5b7fe640 1 --2- 172.21.15.83:0/3318712718 >> v2:172.21.15.79:3300/0 conn(0x7fab5c06feb0 0x7fab5c071af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.154+0000 7fab5b7fe640 1 -- 172.21.15.83:0/3318712718 >> v1:172.21.15.83:6789/0 conn(0x7fab5c0d1310 legacy=0x7fab5c06f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.164 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.154+0000 7fab5affd640 1 -- 172.21.15.83:0/3318712718 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4211303649 0 0) 0x7fab5c0cee20 con 0x7fab5c0d1310 2024-04-26T19:50:08.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.154+0000 7fab5b7fe640 1 -- 172.21.15.83:0/3318712718 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c0ce7d0 con 0x7fab5c15b810 2024-04-26T19:50:08.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.154+0000 7fab5b7fe640 1 --2- 172.21.15.83:0/3318712718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 0x7fab5c15f440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab5c070ee0 tx=0x7fab4c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2288feb016242fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:08.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.154+0000 7fab5affd640 1 -- 172.21.15.83:0/3318712718 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c037070 con 0x7fab5c15b810 2024-04-26T19:50:08.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.154+0000 7fab5affd640 1 -- 172.21.15.83:0/3318712718 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab4c032440 con 0x7fab5c15b810 2024-04-26T19:50:08.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.154+0000 7fab5affd640 1 -- 172.21.15.83:0/3318712718 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4c032810 con 0x7fab5c15b810 2024-04-26T19:50:08.165 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.155+0000 7fab62146640 1 -- 172.21.15.83:0/3318712718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 msgr2=0x7fab5c15f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.155+0000 7fab62146640 1 --2- 172.21.15.83:0/3318712718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 0x7fab5c15f440 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fab5c070ee0 tx=0x7fab4c02a1a0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.155+0000 7fab62146640 1 -- 172.21.15.83:0/3318712718 shutdown_connections 2024-04-26T19:50:08.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.155+0000 7fab62146640 1 --2- 172.21.15.83:0/3318712718 >> v2:172.21.15.79:3300/0 conn(0x7fab5c06feb0 0x7fab5c071af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.155+0000 7fab62146640 1 --2- 172.21.15.83:0/3318712718 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 0x7fab5c15f440 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.155+0000 7fab62146640 1 -- 172.21.15.83:0/3318712718 >> 172.21.15.83:0/3318712718 conn(0x7fab5c0b42f0 msgr2=0x7fab5c0b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:08.166 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.155+0000 7fab62146640 1 -- 172.21.15.83:0/3318712718 shutdown_connections 2024-04-26T19:50:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.155+0000 7fab62146640 1 -- 172.21.15.83:0/3318712718 wait complete. 2024-04-26T19:50:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.156+0000 7fab62146640 1 Processor -- start 2024-04-26T19:50:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.156+0000 7fab62146640 1 -- start start 2024-04-26T19:50:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab62146640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab5c06feb0 0x7fab5c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab62146640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab5c0d1310 0x7fab5c149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.167 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab62146640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 0x7fab5c14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab62146640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c15bbf0 con 0x7fab5c06feb0 2024-04-26T19:50:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab62146640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fab5c15f980 con 0x7fab5c15b810 2024-04-26T19:50:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab62146640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fab5c0b0ad0 con 0x7fab5c0d1310 2024-04-26T19:50:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 0x7fab5c14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab53fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab5c0d1310 0x7fab5c149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.168 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5bfff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 0x7fab5c14e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.83:44382/0 (socket says 172.21.15.83:44382) 2024-04-26T19:50:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5bfff640 1 -- 172.21.15.83:0/2770959322 learned_addr learned my addr 172.21.15.83:0/2770959322 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5b7fe640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab5c06feb0 0x7fab5c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5bfff640 1 -- 172.21.15.83:0/2770959322 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab5c0d1310 msgr2=0x7fab5c149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5bfff640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab5c0d1310 0x7fab5c149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5bfff640 1 -- 172.21.15.83:0/2770959322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab5c06feb0 msgr2=0x7fab5c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.169 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5bfff640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab5c06feb0 0x7fab5c156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5bfff640 1 -- 172.21.15.83:0/2770959322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5c138e30 con 0x7fab5c15b810 2024-04-26T19:50:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab5b7fe640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab5c06feb0 0x7fab5c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.157+0000 7fab53fff640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab5c0d1310 0x7fab5c149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.158+0000 7fab5bfff640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 0x7fab5c14e610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab5400efc0 tx=0x7fab5400c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:08.170 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.158+0000 7fab58ff9640 1 -- 172.21.15.83:0/2770959322 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54018020 con 0x7fab5c15b810 2024-04-26T19:50:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.158+0000 7fab58ff9640 1 -- 172.21.15.83:0/2770959322 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab54014480 con 0x7fab5c15b810 2024-04-26T19:50:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.158+0000 7fab58ff9640 1 -- 172.21.15.83:0/2770959322 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54009e60 con 0x7fab5c15b810 2024-04-26T19:50:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.158+0000 7fab62146640 1 -- 172.21.15.83:0/2770959322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5c139110 con 0x7fab5c15b810 2024-04-26T19:50:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.158+0000 7fab62146640 1 -- 172.21.15.83:0/2770959322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5c1396d0 con 0x7fab5c15b810 2024-04-26T19:50:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.158+0000 7fab58ff9640 1 -- 172.21.15.83:0/2770959322 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab5401b020 con 0x7fab5c15b810 2024-04-26T19:50:08.171 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.158+0000 7fab58ff9640 1 -- 172.21.15.83:0/2770959322 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fab5401ddb0 con 0x7fab5c15b810 2024-04-26T19:50:08.172 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.159+0000 7fab62146640 1 -- 172.21.15.83:0/2770959322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5c0506c0 con 0x7fab5c15b810 2024-04-26T19:50:08.172 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:08.160+0000 7fdb97d21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:50:08.173 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.161+0000 7fab58ff9640 1 -- 172.21.15.83:0/2770959322 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fab54022020 con 0x7fab5c15b810 2024-04-26T19:50:08.234 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.233+0000 7fab62146640 1 -- 172.21.15.83:0/2770959322 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "baf0c717-e2cd-4c39-aea1-5b69fab094f3", "id": 6} v 0) -- 0x7fab5c0b17f0 con 0x7fab5c15b810 2024-04-26T19:50:08.243 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.242+0000 7fab58ff9640 1 -- 172.21.15.83:0/2770959322 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "baf0c717-e2cd-4c39-aea1-5b69fab094f3", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fab54010040 con 0x7fab5c15b810 2024-04-26T19:50:08.243 INFO:teuthology.orchestra.run.smithi083.stdout:6 2024-04-26T19:50:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.243+0000 7fab62146640 1 -- 172.21.15.83:0/2770959322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 msgr2=0x7fab5c14e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.243+0000 7fab62146640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 0x7fab5c14e610 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab5400efc0 tx=0x7fab5400c490 comp rx=0 tx=0).stop 2024-04-26T19:50:08.244 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.243+0000 7fab62146640 1 -- 172.21.15.83:0/2770959322 shutdown_connections 2024-04-26T19:50:08.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.243+0000 7fab62146640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fab5c15b810 0x7fab5c14e610 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.243+0000 7fab62146640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fab5c0d1310 0x7fab5c149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.243+0000 7fab62146640 1 --2- 172.21.15.83:0/2770959322 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fab5c06feb0 0x7fab5c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.243+0000 7fab62146640 1 -- 172.21.15.83:0/2770959322 >> 172.21.15.83:0/2770959322 conn(0x7fab5c0b42f0 msgr2=0x7fab5c06ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:08.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.243+0000 7fab62146640 1 -- 172.21.15.83:0/2770959322 shutdown_connections 2024-04-26T19:50:08.245 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.243+0000 7fab62146640 1 -- 172.21.15.83:0/2770959322 wait complete. 2024-04-26T19:50:08.255 DEBUG:teuthology.orchestra.run.smithi083:> sudo ceph --cluster ceph osd new 65254695-e7da-47cc-90a7-8a2f9ca0ce36 7 2024-04-26T19:50:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.351+0000 7f8ec36bb640 1 Processor -- start 2024-04-26T19:50:08.352 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.351+0000 7f8ec36bb640 1 -- start start 2024-04-26T19:50:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec36bb640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8ebc0ab730 0x7f8ebc0b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec36bb640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc0d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec36bb640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ebc15e7e0 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec36bb640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f8ebc0abb10 con 0x7f8ebc0ab730 2024-04-26T19:50:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec36bb640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8ebc161cd0 con 0x7f8ebc15e1c0 2024-04-26T19:50:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec1c58640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc0d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec1c58640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc0d1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.83:56400/0 (socket says 172.21.15.83:56400) 2024-04-26T19:50:08.354 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec1c58640 1 -- 172.21.15.83:0/1255116691 learned_addr learned my addr 172.21.15.83:0/1255116691 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec1457640 1 --2- 172.21.15.83:0/1255116691 >> v2:172.21.15.79:3300/0 conn(0x7f8ebc0ab730 0x7f8ebc0b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec1c58640 1 -- 172.21.15.83:0/1255116691 >> v2:172.21.15.79:3300/0 conn(0x7f8ebc0ab730 msgr2=0x7f8ebc0b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec1c58640 1 --2- 172.21.15.83:0/1255116691 >> v2:172.21.15.79:3300/0 conn(0x7f8ebc0ab730 0x7f8ebc0b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec1c58640 1 -- 172.21.15.83:0/1255116691 >> v1:172.21.15.79:6789/0 conn(0x7f8ebc0b3d90 legacy=0x7f8ebc15dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec1c58640 1 -- 172.21.15.83:0/1255116691 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc160f10 con 0x7f8ebc15e1c0 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.352+0000 7f8ec1457640 1 --2- 172.21.15.83:0/1255116691 >> v2:172.21.15.79:3300/0 conn(0x7f8ebc0ab730 0x7f8ebc0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8ec1c58640 1 --2- 172.21.15.83:0/1255116691 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc0d1d60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009e50 tx=0x7f8eb402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c84e2546e3ad797a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8eb3fff640 1 -- 172.21.15.83:0/1255116691 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4036070 con 0x7f8ebc15e1c0 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8eb3fff640 1 -- 172.21.15.83:0/1255116691 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb4031600 con 0x7f8ebc15e1c0 2024-04-26T19:50:08.355 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8eb3fff640 1 -- 172.21.15.83:0/1255116691 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40319d0 con 0x7f8ebc15e1c0 2024-04-26T19:50:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1255116691 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 msgr2=0x7f8ebc0d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8ec36bb640 1 --2- 172.21.15.83:0/1255116691 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc0d1d60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8eb4009e50 tx=0x7f8eb402a710 comp rx=0 tx=0).stop 2024-04-26T19:50:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1255116691 shutdown_connections 2024-04-26T19:50:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8ec36bb640 1 --2- 172.21.15.83:0/1255116691 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc0d1d60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.358 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8ec36bb640 1 --2- 172.21.15.83:0/1255116691 >> v2:172.21.15.79:3300/0 conn(0x7f8ebc0ab730 0x7f8ebc0b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.353+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1255116691 >> 172.21.15.83:0/1255116691 conn(0x7f8ebc073b40 msgr2=0x7f8ebc073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.354+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1255116691 shutdown_connections 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.354+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1255116691 wait complete. 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.355+0000 7f8ec36bb640 1 Processor -- start 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.355+0000 7f8ec36bb640 1 -- start start 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.355+0000 7f8ec36bb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ebc0ab730 0x7f8ebc0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.355+0000 7f8ec36bb640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ebc0b3d90 0x7f8ebc13b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.355+0000 7f8ec36bb640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc1f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.355+0000 7f8ec36bb640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc1617b0 con 0x7f8ebc0ab730 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.355+0000 7f8ec36bb640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ebc1614d0 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.355+0000 7f8ec36bb640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8ebc0d22a0 con 0x7f8ebc15e1c0 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec1c58640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc1f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec1c58640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc1f36e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.83:56410/0 (socket says 172.21.15.83:56410) 2024-04-26T19:50:08.359 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec1c58640 1 -- 172.21.15.83:0/1970085083 learned_addr learned my addr 172.21.15.83:0/1970085083 (peer_addr_for_me v2:172.21.15.83:0/0) 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec0c56640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ebc0b3d90 0x7f8ebc13b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec1457640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ebc0ab730 0x7f8ebc0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec0c56640 1 -- 172.21.15.83:0/1970085083 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 msgr2=0x7f8ebc1f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec0c56640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc1f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec0c56640 1 -- 172.21.15.83:0/1970085083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ebc0ab730 msgr2=0x7f8ebc0adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec0c56640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ebc0ab730 0x7f8ebc0adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec0c56640 1 -- 172.21.15.83:0/1970085083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ebc0af190 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec1c58640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc1f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec0c56640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ebc0b3d90 0x7f8ebc13b520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800c340 tx=0x7f8eb800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/1970085083 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb80052d0 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/1970085083 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eb8005490 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1970085083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ebc0ae6a0 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/1970085083 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eb8007260 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.360 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.356+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/1970085083 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8eb8007400 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.357+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1970085083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ebc0af430 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.357+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/1970085083 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f8eb80035f0 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.357+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1970085083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ebc0ac5c0 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.361 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.359+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/1970085083 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f8ebc0ac5c0 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.411 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:08.410+0000 7fb7d1810040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:50:08.433 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.432+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1970085083 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd new", "uuid": "65254695-e7da-47cc-90a7-8a2f9ca0ce36", "id": 7} v 0) -- 0x7f8ebc0506c0 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.438 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.437+0000 7f8eb1ffb640 1 -- 172.21.15.83:0/1970085083 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "65254695-e7da-47cc-90a7-8a2f9ca0ce36", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f8eb8003b40 con 0x7f8ebc0b3d90 2024-04-26T19:50:08.439 INFO:teuthology.orchestra.run.smithi083.stdout:7 2024-04-26T19:50:08.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.438+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1970085083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ebc0b3d90 msgr2=0x7f8ebc13b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.438+0000 7f8ec36bb640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ebc0b3d90 0x7f8ebc13b520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8eb800c340 tx=0x7f8eb800c800 comp rx=0 tx=0).stop 2024-04-26T19:50:08.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.438+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1970085083 shutdown_connections 2024-04-26T19:50:08.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.438+0000 7f8ec36bb640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ebc15e1c0 0x7f8ebc1f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.438+0000 7f8ec36bb640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ebc0b3d90 0x7f8ebc13b520 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.438+0000 7f8ec36bb640 1 --2- 172.21.15.83:0/1970085083 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ebc0ab730 0x7f8ebc0adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.438+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1970085083 >> 172.21.15.83:0/1970085083 conn(0x7f8ebc073b40 msgr2=0x7f8ebc15ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:08.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.438+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1970085083 shutdown_connections 2024-04-26T19:50:08.442 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:50:08.438+0000 7f8ec36bb640 1 -- 172.21.15.83:0/1970085083 wait complete. 2024-04-26T19:50:08.451 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T19:50:08.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T19:50:08.453 INFO:tasks.ceph.osd.0:Started 2024-04-26T19:50:08.453 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T19:50:08.453 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T19:50:08.462 INFO:tasks.ceph.osd.1:Started 2024-04-26T19:50:08.462 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T19:50:08.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T19:50:08.466 INFO:tasks.ceph.osd.2:Started 2024-04-26T19:50:08.467 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-26T19:50:08.467 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-26T19:50:08.469 INFO:tasks.ceph.osd.3:Started 2024-04-26T19:50:08.469 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-26T19:50:08.469 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-26T19:50:08.472 INFO:tasks.ceph.osd.4:Started 2024-04-26T19:50:08.472 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-26T19:50:08.472 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-26T19:50:08.475 INFO:tasks.ceph.osd.5:Started 2024-04-26T19:50:08.475 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-26T19:50:08.475 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-26T19:50:08.478 INFO:tasks.ceph.osd.6:Started 2024-04-26T19:50:08.478 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-26T19:50:08.478 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-26T19:50:08.480 INFO:tasks.ceph.osd.7:Started 2024-04-26T19:50:08.481 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:08.589 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:08.588+0000 7fdb97d21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:50:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.598+0000 7f0e0bf1b640 1 Processor -- start 2024-04-26T19:50:08.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.598+0000 7f0e0bf1b640 1 -- start start 2024-04-26T19:50:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.599+0000 7f0e0bf1b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0e040d1b40 0x7f0e0406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.599+0000 7f0e0bf1b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0e040c6d50 0x7f0e040c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.599+0000 7f0e0bf1b640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e040c7a40 con 0x7f0e040b4b90 2024-04-26T19:50:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.599+0000 7f0e0bf1b640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f0e040700c0 con 0x7f0e040d1b40 2024-04-26T19:50:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.599+0000 7f0e0bf1b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0e0406f0a0 con 0x7f0e040c6d50 2024-04-26T19:50:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.599+0000 7f0e094b6640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0e040d1b40 0x7f0e0406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.599+0000 7f0e094b6640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0e040d1b40 0x7f0e0406fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37198/0 (socket says 172.21.15.79:37198) 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.599+0000 7f0e094b6640 1 -- 172.21.15.79:0/869503934 learned_addr learned my addr 172.21.15.79:0/869503934 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.599+0000 7f0e0a4b8640 1 --2- 172.21.15.79:0/869503934 >> v2:172.21.15.83:3300/0 conn(0x7f0e040c6d50 0x7f0e040c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.600+0000 7f0e094b6640 1 -- 172.21.15.79:0/869503934 >> v2:172.21.15.83:3300/0 conn(0x7f0e040c6d50 msgr2=0x7f0e040c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.600+0000 7f0e094b6640 1 --2- 172.21.15.79:0/869503934 >> v2:172.21.15.83:3300/0 conn(0x7f0e040c6d50 0x7f0e040c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.600+0000 7f0e08cb5640 1 -- 172.21.15.79:0/869503934 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2178704666 0 0) 0x7f0e040c7a40 con 0x7f0e040b4b90 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.600+0000 7f0e094b6640 1 -- 172.21.15.79:0/869503934 >> v1:172.21.15.79:6789/0 conn(0x7f0e040b4b90 legacy=0x7f0e040d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.600+0000 7f0e094b6640 1 -- 172.21.15.79:0/869503934 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0406f380 con 0x7f0e040d1b40 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.600+0000 7f0e094b6640 1 --2- 172.21.15.79:0/869503934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040d1b40 0x7f0e0406fb80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0df800bb10 tx=0x7f0df802c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f039fa4f1db428 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e08cb5640 1 -- 172.21.15.79:0/869503934 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8036020 con 0x7f0e040d1b40 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e08cb5640 1 -- 172.21.15.79:0/869503934 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df8032820 con 0x7f0e040d1b40 2024-04-26T19:50:08.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e08cb5640 1 -- 172.21.15.79:0/869503934 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df8032bf0 con 0x7f0e040d1b40 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/869503934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040d1b40 msgr2=0x7f0e0406fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e0bf1b640 1 --2- 172.21.15.79:0/869503934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040d1b40 0x7f0e0406fb80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0df800bb10 tx=0x7f0df802c960 comp rx=0 tx=0).stop 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/869503934 shutdown_connections 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e0bf1b640 1 --2- 172.21.15.79:0/869503934 >> v2:172.21.15.83:3300/0 conn(0x7f0e040c6d50 0x7f0e040c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e0bf1b640 1 --2- 172.21.15.79:0/869503934 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040d1b40 0x7f0e0406fb80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/869503934 >> 172.21.15.79:0/869503934 conn(0x7f0e04073820 msgr2=0x7f0e040d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/869503934 shutdown_connections 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.601+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/869503934 wait complete. 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.602+0000 7f0e0bf1b640 1 Processor -- start 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.602+0000 7f0e0bf1b640 1 -- start start 2024-04-26T19:50:08.606 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e0bf1b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040b4b90 0x7f0e040c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e0bf1b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e040c6d50 0x7f0e041604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e0bf1b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0e040d1b40 0x7f0e04146580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e0bf1b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0e040c7d20 con 0x7f0e040b4b90 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e0bf1b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0e040c7720 con 0x7f0e040c6d50 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e0bf1b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0e040b4f70 con 0x7f0e040d1b40 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e09cb7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040b4b90 0x7f0e040c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e094b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e040c6d50 0x7f0e041604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e094b6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e040c6d50 0x7f0e041604b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:36394/0 (socket says 172.21.15.79:36394) 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e094b6640 1 -- 172.21.15.79:0/1430840389 learned_addr learned my addr 172.21.15.79:0/1430840389 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.603+0000 7f0e0a4b8640 1 --2- 172.21.15.79:0/1430840389 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0e040d1b40 0x7f0e04146580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0e09cb7640 1 -- 172.21.15.79:0/1430840389 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0e040d1b40 msgr2=0x7f0e04146580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0e09cb7640 1 --2- 172.21.15.79:0/1430840389 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0e040d1b40 0x7f0e04146580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0e09cb7640 1 -- 172.21.15.79:0/1430840389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e040c6d50 msgr2=0x7f0e041604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.607 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0e09cb7640 1 --2- 172.21.15.79:0/1430840389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e040c6d50 0x7f0e041604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0e09cb7640 1 -- 172.21.15.79:0/1430840389 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0415a160 con 0x7f0e040b4b90 2024-04-26T19:50:08.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0e094b6640 1 --2- 172.21.15.79:0/1430840389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e040c6d50 0x7f0e041604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:08.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0e09cb7640 1 --2- 172.21.15.79:0/1430840389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040b4b90 0x7f0e040c5a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c3e0 tx=0x7f0e0000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:08.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0df67fc640 1 -- 172.21.15.79:0/1430840389 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00017070 con 0x7f0e040b4b90 2024-04-26T19:50:08.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/1430840389 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0415a440 con 0x7f0e040b4b90 2024-04-26T19:50:08.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0df67fc640 1 -- 172.21.15.79:0/1430840389 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e000035f0 con 0x7f0e040b4b90 2024-04-26T19:50:08.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0df67fc640 1 -- 172.21.15.79:0/1430840389 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00017070 con 0x7f0e040b4b90 2024-04-26T19:50:08.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/1430840389 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0415aa00 con 0x7f0e040b4b90 2024-04-26T19:50:08.608 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0df67fc640 1 -- 172.21.15.79:0/1430840389 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e0001a070 con 0x7f0e040b4b90 2024-04-26T19:50:08.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.604+0000 7f0df67fc640 1 -- 172.21.15.79:0/1430840389 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0e000037a0 con 0x7f0e040b4b90 2024-04-26T19:50:08.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.605+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/1430840389 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e040506c0 con 0x7f0e040b4b90 2024-04-26T19:50:08.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.607+0000 7f0df67fc640 1 -- 172.21.15.79:0/1430840389 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f0e000039a0 con 0x7f0e040b4b90 2024-04-26T19:50:08.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.668+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/1430840389 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e04070b90 con 0x7f0e040b4b90 2024-04-26T19:50:08.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.668+0000 7f0df67fc640 1 -- 172.21.15.79:0/1430840389 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0e000039a0 con 0x7f0e040b4b90 2024-04-26T19:50:08.669 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:08.669 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":10,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:08.434954+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:08.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.669+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/1430840389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040b4b90 msgr2=0x7f0e040c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:08.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.669+0000 7f0e0bf1b640 1 --2- 172.21.15.79:0/1430840389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040b4b90 0x7f0e040c5a20 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c3e0 tx=0x7f0e0000c8a0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.669+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/1430840389 shutdown_connections 2024-04-26T19:50:08.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.669+0000 7f0e0bf1b640 1 --2- 172.21.15.79:0/1430840389 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0e040d1b40 0x7f0e04146580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.669+0000 7f0e0bf1b640 1 --2- 172.21.15.79:0/1430840389 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0e040c6d50 0x7f0e041604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.669+0000 7f0e0bf1b640 1 --2- 172.21.15.79:0/1430840389 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0e040b4b90 0x7f0e040c5a20 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:08.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.669+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/1430840389 >> 172.21.15.79:0/1430840389 conn(0x7f0e04073820 msgr2=0x7f0e0415e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:08.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.669+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/1430840389 shutdown_connections 2024-04-26T19:50:08.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:08.669+0000 7f0e0bf1b640 1 -- 172.21.15.79:0/1430840389 wait complete. 2024-04-26T19:50:08.681 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T19:50:08.681 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T19:50:08.700 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:08.700+0000 7fb7d1810040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:50:08.751 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:08.751+0000 7fb7d1810040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:50:08.824 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-04-26T19:50:08.823+0000 7f56c0dcb640 -1 Falling back to public interface 2024-04-26T19:50:08.847 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:08.846+0000 7fdb97d21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:50:08.869 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:08.868+0000 7fb7d1810040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:50:08.899 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:08.899+0000 7fdb97d21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:50:09.013 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:09.013+0000 7fdb97d21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:50:09.073 INFO:tasks.ceph.osd.6.smithi083.stderr:2024-04-26T19:50:09.073+0000 7fa229568640 -1 Falling back to public interface 2024-04-26T19:50:09.074 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-04-26T19:50:09.073+0000 7f3140c0f640 -1 Falling back to public interface 2024-04-26T19:50:09.074 INFO:tasks.ceph.osd.4.smithi083.stderr:2024-04-26T19:50:09.074+0000 7f0d26c0e640 -1 Falling back to public interface 2024-04-26T19:50:09.078 INFO:tasks.ceph.osd.7.smithi083.stderr:2024-04-26T19:50:09.078+0000 7f5c6869a640 -1 Falling back to public interface 2024-04-26T19:50:09.082 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-04-26T19:50:09.081+0000 7f2b0f9d6640 -1 Falling back to public interface 2024-04-26T19:50:09.084 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-04-26T19:50:09.083+0000 7f822e21e640 -1 Falling back to public interface 2024-04-26T19:50:09.088 INFO:tasks.ceph.osd.5.smithi083.stderr:2024-04-26T19:50:09.088+0000 7f8d21568640 -1 Falling back to public interface 2024-04-26T19:50:09.149 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:09.148+0000 7fb7d1810040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:50:09.195 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:09.195+0000 7fb7d1810040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:50:09.297 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:09.297+0000 7fdb97d21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:50:09.344 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:09.344+0000 7fdb97d21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:50:09.350 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:09.349+0000 7fb7d1810040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:50:09.485 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:09.484+0000 7fb7d1810040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:50:09.487 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:09.499 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:09.498+0000 7fdb97d21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:50:09.536 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:09.536+0000 7fb7d1810040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:50:09.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.576+0000 7f7dbd5c9640 1 Processor -- start 2024-04-26T19:50:09.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.576+0000 7f7dbd5c9640 1 -- start start 2024-04-26T19:50:09.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.576+0000 7f7dbd5c9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7db80b6880 0x7f7db80d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:09.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.576+0000 7f7dbd5c9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:09.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.576+0000 7f7dbd5c9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db80d2450 con 0x7f7db8162980 2024-04-26T19:50:09.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.576+0000 7f7dbd5c9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7db80cfbd0 con 0x7f7db80b6880 2024-04-26T19:50:09.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.576+0000 7f7dbd5c9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f7db80d6120 con 0x7f7db80d1050 2024-04-26T19:50:09.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.577+0000 7f7db77fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:09.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.577+0000 7f7daffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7db80b6880 0x7f7db80d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:09.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.577+0000 7f7db6ffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f7db8162980 0x7f7db80d1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:47812/0 (socket says 172.21.15.79:47812) 2024-04-26T19:50:09.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.577+0000 7f7db6ffd640 1 -- 172.21.15.79:0/2891206251 learned_addr learned my addr 172.21.15.79:0/2891206251 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:09.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.577+0000 7f7db77fe640 1 -- 172.21.15.79:0/2891206251 >> v2:172.21.15.83:3300/0 conn(0x7f7db80b6880 msgr2=0x7f7db80d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:09.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.577+0000 7f7db77fe640 1 --2- 172.21.15.79:0/2891206251 >> v2:172.21.15.83:3300/0 conn(0x7f7db80b6880 0x7f7db80d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.577+0000 7f7db77fe640 1 -- 172.21.15.79:0/2891206251 >> v1:172.21.15.83:6789/0 conn(0x7f7db8162980 legacy=0x7f7db80d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:09.580 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.577+0000 7f7db77fe640 1 -- 172.21.15.79:0/2891206251 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db80d5e40 con 0x7f7db80d1050 2024-04-26T19:50:09.580 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.577+0000 7f7daffff640 1 --2- 172.21.15.79:0/2891206251 >> v2:172.21.15.83:3300/0 conn(0x7f7db80b6880 0x7f7db80d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:09.585 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.578+0000 7f7db77fe640 1 --2- 172.21.15.79:0/2891206251 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80b47b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009d90 tx=0x7f7db002a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3d964e2bba1d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:09.585 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.578+0000 7f7db67fc640 1 -- 172.21.15.79:0/2891206251 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0031600 con 0x7f7db80d1050 2024-04-26T19:50:09.585 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.578+0000 7f7db67fc640 1 -- 172.21.15.79:0/2891206251 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db00317a0 con 0x7f7db80d1050 2024-04-26T19:50:09.585 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.578+0000 7f7db67fc640 1 -- 172.21.15.79:0/2891206251 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db0031b70 con 0x7f7db80d1050 2024-04-26T19:50:09.585 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.579+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/2891206251 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 msgr2=0x7f7db80b47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:09.585 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.579+0000 7f7dbd5c9640 1 --2- 172.21.15.79:0/2891206251 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80b47b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009d90 tx=0x7f7db002a5d0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.579+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/2891206251 shutdown_connections 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.579+0000 7f7dbd5c9640 1 --2- 172.21.15.79:0/2891206251 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80b47b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.579+0000 7f7dbd5c9640 1 --2- 172.21.15.79:0/2891206251 >> v2:172.21.15.83:3300/0 conn(0x7f7db80b6880 0x7f7db80d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.579+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/2891206251 >> 172.21.15.79:0/2891206251 conn(0x7f7db809e520 msgr2=0x7f7db80d51f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.579+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/2891206251 shutdown_connections 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.579+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/2891206251 wait complete. 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.580+0000 7f7dbd5c9640 1 Processor -- start 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.580+0000 7f7dbd5c9640 1 -- start start 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.580+0000 7f7dbd5c9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80b6880 0x7f7db815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.580+0000 7f7dbd5c9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7dbd5c9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db8162980 0x7f7db80cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:09.586 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7dbd5c9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80b5f00 con 0x7f7db80b6880 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7dbd5c9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7db80b5d80 con 0x7f7db8162980 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7dbd5c9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f7db80b4cf0 con 0x7f7db80d1050 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7db6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80b6880 0x7f7db815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7db6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80b6880 0x7f7db815c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37204/0 (socket says 172.21.15.79:37204) 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daffff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7db77fe640 1 --2- 172.21.15.79:0/3237311345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db8162980 0x7f7db80cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7db6ffd640 1 -- 172.21.15.79:0/3237311345 learned_addr learned my addr 172.21.15.79:0/3237311345 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daffff640 1 -- 172.21.15.79:0/3237311345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db8162980 msgr2=0x7f7db80cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daffff640 1 --2- 172.21.15.79:0/3237311345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db8162980 0x7f7db80cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daffff640 1 -- 172.21.15.79:0/3237311345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80b6880 msgr2=0x7f7db815c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daffff640 1 --2- 172.21.15.79:0/3237311345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80b6880 0x7f7db815c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daffff640 1 -- 172.21.15.79:0/3237311345 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db81f4ba0 con 0x7f7db80d1050 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daffff640 1 --2- 172.21.15.79:0/3237311345 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80cb9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7da000e900 tx=0x7f7da000edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:09.587 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daf7fe640 1 -- 172.21.15.79:0/3237311345 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da0017070 con 0x7f7db80d1050 2024-04-26T19:50:09.588 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daf7fe640 1 -- 172.21.15.79:0/3237311345 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da000ca00 con 0x7f7db80d1050 2024-04-26T19:50:09.588 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/3237311345 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db80ca940 con 0x7f7db80d1050 2024-04-26T19:50:09.588 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7daf7fe640 1 -- 172.21.15.79:0/3237311345 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da000cdf0 con 0x7f7db80d1050 2024-04-26T19:50:09.588 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.581+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/3237311345 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db80caf00 con 0x7f7db80d1050 2024-04-26T19:50:09.588 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.582+0000 7f7daf7fe640 1 -- 172.21.15.79:0/3237311345 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7da001a070 con 0x7f7db80d1050 2024-04-26T19:50:09.588 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.582+0000 7f7daf7fe640 1 -- 172.21.15.79:0/3237311345 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7da0013b40 con 0x7f7db80d1050 2024-04-26T19:50:09.588 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.582+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/3237311345 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db8069380 con 0x7f7db80d1050 2024-04-26T19:50:09.588 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.584+0000 7f7daf7fe640 1 -- 172.21.15.79:0/3237311345 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f7da0013d40 con 0x7f7db80d1050 2024-04-26T19:50:09.609 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:09.608+0000 7fb7d1810040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:50:09.634 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:09.634+0000 7fdb97d21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:50:09.647 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.646+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/3237311345 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7db806d450 con 0x7f7db80d1050 2024-04-26T19:50:09.648 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.647+0000 7f7daf7fe640 1 -- 172.21.15.79:0/3237311345 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7da00093a0 con 0x7f7db80d1050 2024-04-26T19:50:09.649 INFO:teuthology.misc.health.smithi079.stdout: 2024-04-26T19:50:09.649 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":10,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:08.434954+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:09.650 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.648+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/3237311345 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 msgr2=0x7f7db80cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:09.650 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.648+0000 7f7dbd5c9640 1 --2- 172.21.15.79:0/3237311345 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80cb9e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7da000e900 tx=0x7f7da000edc0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.650 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.648+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/3237311345 shutdown_connections 2024-04-26T19:50:09.650 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.648+0000 7f7dbd5c9640 1 --2- 172.21.15.79:0/3237311345 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7db8162980 0x7f7db80cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.650 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.648+0000 7f7dbd5c9640 1 --2- 172.21.15.79:0/3237311345 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7db80d1050 0x7f7db80cb9e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.651 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.648+0000 7f7dbd5c9640 1 --2- 172.21.15.79:0/3237311345 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7db80b6880 0x7f7db815c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:09.651 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.648+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/3237311345 >> 172.21.15.79:0/3237311345 conn(0x7f7db809e520 msgr2=0x7f7db806e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:09.651 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.648+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/3237311345 shutdown_connections 2024-04-26T19:50:09.651 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:09.648+0000 7f7dbd5c9640 1 -- 172.21.15.79:0/3237311345 wait complete. 2024-04-26T19:50:09.657 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:09.656+0000 7fb7d1810040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:50:09.659 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T19:50:09.685 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:09.685+0000 7fdb97d21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:50:09.753 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:09.753+0000 7fdb97d21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:50:09.797 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:09.797+0000 7fdb97d21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:50:10.126 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:10.125+0000 7fb7d1810040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:50:10.243 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:10.242+0000 7fb7d1810040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:50:10.282 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:10.280+0000 7fdb97d21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:50:10.393 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:50:10.393+0000 7fdb97d21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:50:11.255 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:11.255+0000 7fb771745640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T19:50:11.256 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:50:11.256+0000 7fb771745640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T19:50:12.340 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-04-26T19:50:12.339+0000 7f56c0dcb640 -1 osd.0 0 log_to_monitors true 2024-04-26T19:50:12.530 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-04-26T19:50:12.529+0000 7f3140c0f640 -1 osd.2 0 log_to_monitors true 2024-04-26T19:50:12.566 INFO:tasks.ceph.osd.7.smithi083.stderr:2024-04-26T19:50:12.566+0000 7f5c6869a640 -1 osd.7 0 log_to_monitors true 2024-04-26T19:50:12.591 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-04-26T19:50:12.590+0000 7f822e21e640 -1 osd.1 0 log_to_monitors true 2024-04-26T19:50:12.613 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-04-26T19:50:12.613+0000 7f2b0f9d6640 -1 osd.3 0 log_to_monitors true 2024-04-26T19:50:12.623 INFO:tasks.ceph.osd.6.smithi083.stderr:2024-04-26T19:50:12.623+0000 7fa229568640 -1 osd.6 0 log_to_monitors true 2024-04-26T19:50:12.643 INFO:tasks.ceph.osd.5.smithi083.stderr:2024-04-26T19:50:12.643+0000 7f8d21568640 -1 osd.5 0 log_to_monitors true 2024-04-26T19:50:12.644 INFO:tasks.ceph.osd.4.smithi083.stderr:2024-04-26T19:50:12.644+0000 7f0d26c0e640 -1 osd.4 0 log_to_monitors true 2024-04-26T19:50:16.075 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-04-26T19:50:16.074+0000 7f56bcd61640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T19:50:16.162 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-04-26T19:50:16.162+0000 7f8229c07640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T19:50:16.176 INFO:tasks.ceph.osd.0.smithi079.stderr:2024-04-26T19:50:16.176+0000 7f56b4b81640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:50:16.201 INFO:tasks.ceph.osd.1.smithi079.stderr:2024-04-26T19:50:16.201+0000 7f8221a38640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:50:16.369 INFO:tasks.ceph.osd.6.smithi083.stderr:2024-04-26T19:50:16.369+0000 7fa2254fe640 -1 osd.6 0 waiting for initial osdmap 2024-04-26T19:50:16.384 INFO:tasks.ceph.osd.4.smithi083.stderr:2024-04-26T19:50:16.384+0000 7f0d225ce640 -1 osd.4 0 waiting for initial osdmap 2024-04-26T19:50:16.386 INFO:tasks.ceph.osd.6.smithi083.stderr:2024-04-26T19:50:16.386+0000 7fa21d31e640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:50:16.389 INFO:tasks.ceph.osd.7.smithi083.stderr:2024-04-26T19:50:16.389+0000 7f5c64630640 -1 osd.7 0 waiting for initial osdmap 2024-04-26T19:50:16.391 INFO:tasks.ceph.osd.5.smithi083.stderr:2024-04-26T19:50:16.392+0000 7f8d1d4fe640 -1 osd.5 0 waiting for initial osdmap 2024-04-26T19:50:16.395 INFO:tasks.ceph.osd.4.smithi083.stderr:2024-04-26T19:50:16.395+0000 7f0d1a3ee640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:50:16.403 INFO:tasks.ceph.osd.7.smithi083.stderr:2024-04-26T19:50:16.403+0000 7f5c5c450640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:50:16.406 INFO:tasks.ceph.osd.5.smithi083.stderr:2024-04-26T19:50:16.407+0000 7f8d1531e640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:50:16.466 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:16.569 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.569+0000 7f3cda472640 1 Processor -- start 2024-04-26T19:50:16.570 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.569+0000 7f3cda472640 1 -- start start 2024-04-26T19:50:16.571 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cda472640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd415d310 0x7f3cd40d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:16.571 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cda472640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cd40d0010 0x7f3cd40cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:16.571 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cda472640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cd40d16e0 con 0x7f3cd40704a0 2024-04-26T19:50:16.571 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cda472640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3cd40d19c0 con 0x7f3cd40d0010 2024-04-26T19:50:16.571 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cda472640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd40d1ca0 con 0x7f3cd415d310 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd3fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd415d310 0x7f3cd40d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd8a0f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cd40d0010 0x7f3cd40cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd3fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd415d310 0x7f3cd40d0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:41202/0 (socket says 172.21.15.79:41202) 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd3fff640 1 -- 172.21.15.79:0/2590940001 learned_addr learned my addr 172.21.15.79:0/2590940001 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd8a0f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cd40d0010 0x7f3cd40cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:36544/0 (socket says 172.21.15.79:36544) 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd37fe640 1 --1- 172.21.15.79:0/2590940001 >> v1:172.21.15.83:6789/0 conn(0x7f3cd40704a0 0x7f3cd40cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:47826/0 (socket says 172.21.15.79:47826) 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd3fff640 1 -- 172.21.15.79:0/2590940001 >> v2:172.21.15.83:3300/0 conn(0x7f3cd40d0010 msgr2=0x7f3cd40cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd3fff640 1 --2- 172.21.15.79:0/2590940001 >> v2:172.21.15.83:3300/0 conn(0x7f3cd40d0010 0x7f3cd40cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd3fff640 1 -- 172.21.15.79:0/2590940001 >> v1:172.21.15.83:6789/0 conn(0x7f3cd40704a0 legacy=0x7f3cd40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd3fff640 1 -- 172.21.15.79:0/2590940001 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd40d1f80 con 0x7f3cd415d310 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd2ffd640 1 -- 172.21.15.79:0/2590940001 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1554969565 0 0) 0x7f3cd40d16e0 con 0x7f3cd40704a0 2024-04-26T19:50:16.572 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.570+0000 7f3cd8a0f640 1 --2- 172.21.15.79:0/2590940001 >> v2:172.21.15.83:3300/0 conn(0x7f3cd40d0010 0x7f3cd40cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:16.574 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.571+0000 7f3cd3fff640 1 --2- 172.21.15.79:0/2590940001 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd415d310 0x7f3cd40d0e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40098a0 tx=0x7f3cc402a160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26be641c2043ef22 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:16.574 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cd2ffd640 1 -- 172.21.15.79:0/2590940001 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4037070 con 0x7f3cd415d310 2024-04-26T19:50:16.574 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cd2ffd640 1 -- 172.21.15.79:0/2590940001 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cc4032440 con 0x7f3cd415d310 2024-04-26T19:50:16.574 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cd2ffd640 1 -- 172.21.15.79:0/2590940001 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4032810 con 0x7f3cd415d310 2024-04-26T19:50:16.574 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cda472640 1 -- 172.21.15.79:0/2590940001 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd415d310 msgr2=0x7f3cd40d0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:16.574 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cda472640 1 --2- 172.21.15.79:0/2590940001 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd415d310 0x7f3cd40d0e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40098a0 tx=0x7f3cc402a160 comp rx=0 tx=0).stop 2024-04-26T19:50:16.575 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cda472640 1 -- 172.21.15.79:0/2590940001 shutdown_connections 2024-04-26T19:50:16.575 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cda472640 1 --2- 172.21.15.79:0/2590940001 >> v2:172.21.15.83:3300/0 conn(0x7f3cd40d0010 0x7f3cd40cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:16.575 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cda472640 1 --2- 172.21.15.79:0/2590940001 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd415d310 0x7f3cd40d0e90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:16.575 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cda472640 1 -- 172.21.15.79:0/2590940001 >> 172.21.15.79:0/2590940001 conn(0x7f3cd40d2f90 msgr2=0x7f3cd40d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:16.575 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cda472640 1 -- 172.21.15.79:0/2590940001 shutdown_connections 2024-04-26T19:50:16.575 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.572+0000 7f3cda472640 1 -- 172.21.15.79:0/2590940001 wait complete. 2024-04-26T19:50:16.575 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.574+0000 7f3cda472640 1 Processor -- start 2024-04-26T19:50:16.575 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.574+0000 7f3cda472640 1 -- start start 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.574+0000 7f3cda472640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40704a0 0x7f3cd40c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.574+0000 7f3cda472640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd40d0010 0x7f3cd40cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.574+0000 7f3cda472640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cd415d310 0x7f3cd40c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.574+0000 7f3cda472640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd40ce300 con 0x7f3cd415d310 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.574+0000 7f3cda472640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd415d6f0 con 0x7f3cd40704a0 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.574+0000 7f3cda472640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd40ce480 con 0x7f3cd40d0010 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd8a0f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cd415d310 0x7f3cd40c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd8a0f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cd415d310 0x7f3cd40c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:37312/0 (socket says 172.21.15.79:37312) 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd8a0f640 1 -- 172.21.15.79:0/3623989232 learned_addr learned my addr 172.21.15.79:0/3623989232 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd37fe640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd40d0010 0x7f3cd40cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd3fff640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40704a0 0x7f3cd40c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:16.578 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd8a0f640 1 -- 172.21.15.79:0/3623989232 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd40d0010 msgr2=0x7f3cd40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd8a0f640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd40d0010 0x7f3cd40cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd8a0f640 1 -- 172.21.15.79:0/3623989232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40704a0 msgr2=0x7f3cd40c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd8a0f640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40704a0 0x7f3cd40c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd8a0f640 1 -- 172.21.15.79:0/3623989232 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd40cca70 con 0x7f3cd415d310 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd3fff640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40704a0 0x7f3cd40c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd8a0f640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cd415d310 0x7f3cd40c6ac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00efb0 tx=0x7f3ccc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd0ff9640 1 -- 172.21.15.79:0/3623989232 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc018020 con 0x7f3cd415d310 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd0ff9640 1 -- 172.21.15.79:0/3623989232 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ccc014480 con 0x7f3cd415d310 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cd0ff9640 1 -- 172.21.15.79:0/3623989232 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc014850 con 0x7f3cd415d310 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd40ccd50 con 0x7f3cd415d310 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.575+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd406ef50 con 0x7f3cd415d310 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.577+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd4069380 con 0x7f3cd415d310 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.577+0000 7f3cd0ff9640 1 -- 172.21.15.79:0/3623989232 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7f3ccc005d10 con 0x7f3cd415d310 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.577+0000 7f3cd0ff9640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3cc006c860 0x7f3cc006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.577+0000 7f3cd3fff640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3cc006c860 0x7f3cc006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:16.579 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.577+0000 7f3cd0ff9640 1 -- 172.21.15.79:0/3623989232 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f3ccc08ba30 con 0x7f3cd415d310 2024-04-26T19:50:16.580 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.578+0000 7f3cd3fff640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3cc006c860 0x7f3cc006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4034000 tx=0x7f3cc402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:16.582 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.582+0000 7f3cd0ff9640 1 -- 172.21.15.79:0/3623989232 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ccc022030 con 0x7f3cd415d310 2024-04-26T19:50:16.802 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-04-26T19:50:16.801+0000 7f2b0b96c640 -1 osd.3 0 waiting for initial osdmap 2024-04-26T19:50:16.811 INFO:tasks.ceph.osd.3.smithi079.stderr:2024-04-26T19:50:16.811+0000 7f2b0378c640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:50:16.861 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.861+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cd40cee90 con 0x7f3cd415d310 2024-04-26T19:50:16.862 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.861+0000 7f3cd0ff9640 1 -- 172.21.15.79:0/3623989232 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f3ccc010040 con 0x7f3cd415d310 2024-04-26T19:50:16.862 INFO:teuthology.misc.health.smithi079.stdout: 2024-04-26T19:50:16.862 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":12,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:14.256142+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:16.864 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3cc006c860 msgr2=0x7f3cc006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:16.864 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3cc006c860 0x7f3cc006ed20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4034000 tx=0x7f3cc402e000 comp rx=0 tx=0).stop 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cd415d310 msgr2=0x7f3cd40c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cd415d310 0x7f3cd40c6ac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00efb0 tx=0x7f3ccc00c490 comp rx=0 tx=0).stop 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 shutdown_connections 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3cc006c860 0x7f3cc006ed20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cd415d310 0x7f3cd40c6ac0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cd40d0010 0x7f3cd40cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 --2- 172.21.15.79:0/3623989232 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cd40704a0 0x7f3cd40c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 >> 172.21.15.79:0/3623989232 conn(0x7f3cd40d2f90 msgr2=0x7f3cd40b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 shutdown_connections 2024-04-26T19:50:16.865 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:16.864+0000 7f3cda472640 1 -- 172.21.15.79:0/3623989232 wait complete. 2024-04-26T19:50:16.878 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-26T19:50:17.014 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-04-26T19:50:17.013+0000 7f313c5ff640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T19:50:17.030 INFO:tasks.ceph.osd.2.smithi079.stderr:2024-04-26T19:50:17.028+0000 7f313441f640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T19:50:23.686 DEBUG:teuthology.orchestra.run.smithi079:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:23.785 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.785+0000 7fdc63ccc640 1 Processor -- start 2024-04-26T19:50:23.786 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.785+0000 7fdc63ccc640 1 -- start start 2024-04-26T19:50:23.794 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc63ccc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdc5c073b60 0x7fdc5c06fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:23.794 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc63ccc640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5c0b6f00 con 0x7fdc5c1608a0 2024-04-26T19:50:23.794 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc63ccc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc5c0b6940 con 0x7fdc5c06ca60 2024-04-26T19:50:23.794 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc63ccc640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5c160c80 con 0x7fdc5c073b60 2024-04-26T19:50:23.795 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc61a68640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdc5c1608a0 0x7fdc5c06c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:48038/0 (socket says 172.21.15.79:48038) 2024-04-26T19:50:23.795 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc62269640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdc5c073b60 0x7fdc5c06fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:23.795 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc61a68640 1 -- 172.21.15.79:0/2532146086 learned_addr learned my addr 172.21.15.79:0/2532146086 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:23.795 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc62269640 1 -- 172.21.15.79:0/2532146086 >> v1:172.21.15.83:6789/0 conn(0x7fdc5c06ca60 legacy=0x7fdc5c06f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:23.795 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc62269640 1 -- 172.21.15.79:0/2532146086 >> v1:172.21.15.79:6789/0 conn(0x7fdc5c1608a0 legacy=0x7fdc5c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:23.795 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc62269640 1 -- 172.21.15.79:0/2532146086 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c0b6c20 con 0x7fdc5c073b60 2024-04-26T19:50:23.795 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.786+0000 7fdc60a66640 1 -- 172.21.15.79:0/2532146086 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360813969 0 0) 0x7fdc5c0b6f00 con 0x7fdc5c1608a0 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.787+0000 7fdc62269640 1 --2- 172.21.15.79:0/2532146086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c073b60 0x7fdc5c06fe20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009d90 tx=0x7fdc5402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef93dc33ea700aea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.787+0000 7fdc60a66640 1 -- 172.21.15.79:0/2532146086 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54007740 con 0x7fdc5c073b60 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.787+0000 7fdc60a66640 1 -- 172.21.15.79:0/2532146086 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc540078e0 con 0x7fdc5c073b60 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.787+0000 7fdc60a66640 1 -- 172.21.15.79:0/2532146086 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54007cb0 con 0x7fdc5c073b60 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.788+0000 7fdc63ccc640 1 -- 172.21.15.79:0/2532146086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c073b60 msgr2=0x7fdc5c06fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.788+0000 7fdc63ccc640 1 --2- 172.21.15.79:0/2532146086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c073b60 0x7fdc5c06fe20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009d90 tx=0x7fdc5402a6a0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.788+0000 7fdc63ccc640 1 -- 172.21.15.79:0/2532146086 shutdown_connections 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.788+0000 7fdc63ccc640 1 --2- 172.21.15.79:0/2532146086 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c073b60 0x7fdc5c06fe20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.788+0000 7fdc63ccc640 1 -- 172.21.15.79:0/2532146086 >> 172.21.15.79:0/2532146086 conn(0x7fdc5c0b3bd0 msgr2=0x7fdc5c0b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.788+0000 7fdc63ccc640 1 -- 172.21.15.79:0/2532146086 shutdown_connections 2024-04-26T19:50:23.796 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.788+0000 7fdc63ccc640 1 -- 172.21.15.79:0/2532146086 wait complete. 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.789+0000 7fdc63ccc640 1 Processor -- start 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.789+0000 7fdc63ccc640 1 -- start start 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc63ccc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c06ca60 0x7fdc5c0ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc63ccc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc5c073b60 0x7fdc5c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc63ccc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5c1608a0 0x7fdc5c09de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc63ccc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c0737a0 con 0x7fdc5c06ca60 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc63ccc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c0b6610 con 0x7fdc5c1608a0 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc63ccc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5c073920 con 0x7fdc5c073b60 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc61a68640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c06ca60 0x7fdc5c0ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc61a68640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c06ca60 0x7fdc5c0ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34814/0 (socket says 172.21.15.79:34814) 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc61a68640 1 -- 172.21.15.79:0/1242207206 learned_addr learned my addr 172.21.15.79:0/1242207206 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc61a68640 1 -- 172.21.15.79:0/1242207206 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc5c073b60 msgr2=0x7fdc5c155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc61a68640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc5c073b60 0x7fdc5c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.797 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc61a68640 1 -- 172.21.15.79:0/1242207206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5c1608a0 msgr2=0x7fdc5c09de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc62269640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5c1608a0 0x7fdc5c09de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc61a68640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5c1608a0 0x7fdc5c09de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc61a68640 1 -- 172.21.15.79:0/1242207206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c09c890 con 0x7fdc5c06ca60 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc61a68640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c06ca60 0x7fdc5c0ce0c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c430 tx=0x7fdc5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc527fc640 1 -- 172.21.15.79:0/1242207206 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580074e0 con 0x7fdc5c06ca60 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc527fc640 1 -- 172.21.15.79:0/1242207206 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc580047e0 con 0x7fdc5c06ca60 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c09c2d0 con 0x7fdc5c06ca60 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.790+0000 7fdc527fc640 1 -- 172.21.15.79:0/1242207206 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58004bd0 con 0x7fdc5c06ca60 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.791+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c0af9f0 con 0x7fdc5c06ca60 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.791+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc5c069380 con 0x7fdc5c06ca60 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.792+0000 7fdc527fc640 1 -- 172.21.15.79:0/1242207206 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc58011070 con 0x7fdc5c06ca60 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.792+0000 7fdc527fc640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdc4006c8f0 0x7fdc4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:23.798 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.792+0000 7fdc61267640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdc4006c8f0 0x7fdc4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:23.799 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.792+0000 7fdc527fc640 1 -- 172.21.15.79:0/1242207206 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fdc5808b4e0 con 0x7fdc5c06ca60 2024-04-26T19:50:23.799 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.793+0000 7fdc61267640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdc4006c8f0 0x7fdc4006edb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c158410 tx=0x7fdc4c0079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:23.799 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.797+0000 7fdc527fc640 1 -- 172.21.15.79:0/1242207206 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc58059c60 con 0x7fdc5c06ca60 2024-04-26T19:50:23.971 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.971+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc5c0b5d50 con 0x7fdc5c06ca60 2024-04-26T19:50:23.972 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.972+0000 7fdc527fc640 1 -- 172.21.15.79:0/1242207206 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7fdc58059560 con 0x7fdc5c06ca60 2024-04-26T19:50:23.972 INFO:teuthology.misc.health.smithi079.stdout: 2024-04-26T19:50:23.972 INFO:teuthology.misc.health.smithi079.stdout:{"epoch":16,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:20.295868+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:23.974 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.973+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdc4006c8f0 msgr2=0x7fdc4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.973+0000 7fdc63ccc640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdc4006c8f0 0x7fdc4006edb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c158410 tx=0x7fdc4c0079b0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.973+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c06ca60 msgr2=0x7fdc5c0ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.973+0000 7fdc63ccc640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c06ca60 0x7fdc5c0ce0c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800c430 tx=0x7fdc5800c8f0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.974+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 shutdown_connections 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.974+0000 7fdc63ccc640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdc4006c8f0 0x7fdc4006edb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.974+0000 7fdc63ccc640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc5c1608a0 0x7fdc5c09de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.974+0000 7fdc63ccc640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc5c073b60 0x7fdc5c155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.974+0000 7fdc63ccc640 1 --2- 172.21.15.79:0/1242207206 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc5c06ca60 0x7fdc5c0ce0c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.974+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 >> 172.21.15.79:0/1242207206 conn(0x7fdc5c0b3bd0 msgr2=0x7fdc5c0b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.974+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 shutdown_connections 2024-04-26T19:50:23.975 INFO:teuthology.misc.health.smithi079.stderr:2024-04-26T19:50:23.974+0000 7fdc63ccc640 1 -- 172.21.15.79:0/1242207206 wait complete. 2024-04-26T19:50:23.984 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-26T19:50:23.984 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T19:50:23.984 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T19:50:23.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T19:50:23.987 INFO:tasks.ceph.mds.a:Started 2024-04-26T19:50:23.987 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T19:50:23.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T19:50:23.989 INFO:tasks.ceph.mds.c:Started 2024-04-26T19:50:23.989 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T19:50:23.989 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T19:50:23.996 INFO:tasks.ceph.mds.b:Started 2024-04-26T19:50:23.996 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T19:50:23.996 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T19:50:23.998 INFO:tasks.ceph.mds.d:Started 2024-04-26T19:50:23.999 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-26T19:50:23.999 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:24.060 INFO:tasks.ceph.mds.a.smithi079.stdout:starting mds.a at 2024-04-26T19:50:24.060 INFO:tasks.ceph.mds.c.smithi079.stdout:starting mds.c at 2024-04-26T19:50:24.060 INFO:tasks.ceph.mds.c.smithi079.stderr:2024-04-26T19:50:24.060+0000 7f56b5967640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-26T19:50:24.080 INFO:tasks.ceph.mds.b.smithi083.stdout:starting mds.b at 2024-04-26T19:50:24.080 INFO:tasks.ceph.mds.d.smithi083.stdout:starting mds.d at 2024-04-26T19:50:24.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.101+0000 7f6edd90a640 1 Processor -- start 2024-04-26T19:50:24.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.101+0000 7f6edd90a640 1 -- start start 2024-04-26T19:50:24.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.101+0000 7f6edd90a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160630 0x7f6ed80d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.101+0000 7f6edd90a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6ed8144a80 0x7f6ed80720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.101+0000 7f6edd90a640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ed80cefe0 con 0x7f6ed8160250 2024-04-26T19:50:24.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.101+0000 7f6edd90a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6ed8144ff0 con 0x7f6ed8144a80 2024-04-26T19:50:24.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.101+0000 7f6edd90a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed81452d0 con 0x7f6ed8160630 2024-04-26T19:50:24.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.101+0000 7f6ed6ffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6ed8160250 0x7f6ed80c9130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:48050/0 (socket says 172.21.15.79:48050) 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.101+0000 7f6ed6ffd640 1 -- 172.21.15.79:0/651579622 learned_addr learned my addr 172.21.15.79:0/651579622 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ecffff640 1 --2- 172.21.15.79:0/651579622 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160630 0x7f6ed80d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ed77fe640 1 --2- 172.21.15.79:0/651579622 >> v2:172.21.15.83:3300/0 conn(0x7f6ed8144a80 0x7f6ed80720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ed67fc640 1 -- 172.21.15.79:0/651579622 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1045288003 0 0) 0x7f6ed80cefe0 con 0x7f6ed8160250 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ed67fc640 1 -- 172.21.15.79:0/651579622 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ec4003520 con 0x7f6ed8160250 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ecffff640 1 -- 172.21.15.79:0/651579622 >> v2:172.21.15.83:3300/0 conn(0x7f6ed8144a80 msgr2=0x7f6ed80720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ecffff640 1 --2- 172.21.15.79:0/651579622 >> v2:172.21.15.83:3300/0 conn(0x7f6ed8144a80 0x7f6ed80720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ecffff640 1 -- 172.21.15.79:0/651579622 >> v1:172.21.15.79:6789/0 conn(0x7f6ed8160250 legacy=0x7f6ed80c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ed67fc640 1 -- 172.21.15.79:0/651579622 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 864223235 0 0) 0x7f6ec4003520 con 0x7f6ed8160250 2024-04-26T19:50:24.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ecffff640 1 -- 172.21.15.79:0/651579622 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed81455b0 con 0x7f6ed8160630 2024-04-26T19:50:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.102+0000 7f6ed77fe640 1 --2- 172.21.15.79:0/651579622 >> v2:172.21.15.83:3300/0 conn(0x7f6ed8144a80 0x7f6ed80720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6ecffff640 1 --2- 172.21.15.79:0/651579622 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160630 0x7f6ed80d17d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00097d0 tx=0x7f6ec002a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d66302b8a39262 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6ed67fc640 1 -- 172.21.15.79:0/651579622 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0037070 con 0x7f6ed8160630 2024-04-26T19:50:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6ed67fc640 1 -- 172.21.15.79:0/651579622 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ec002ab40 con 0x7f6ed8160630 2024-04-26T19:50:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6ed67fc640 1 -- 172.21.15.79:0/651579622 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8001690 con 0x7f6ed8160630 2024-04-26T19:50:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6edd90a640 1 -- 172.21.15.79:0/651579622 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160630 msgr2=0x7f6ed80d17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6edd90a640 1 --2- 172.21.15.79:0/651579622 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160630 0x7f6ed80d17d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6ec00097d0 tx=0x7f6ec002a0b0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6edd90a640 1 -- 172.21.15.79:0/651579622 shutdown_connections 2024-04-26T19:50:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6edd90a640 1 --2- 172.21.15.79:0/651579622 >> v2:172.21.15.83:3300/0 conn(0x7f6ed8144a80 0x7f6ed80720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6edd90a640 1 --2- 172.21.15.79:0/651579622 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160630 0x7f6ed80d17d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6edd90a640 1 -- 172.21.15.79:0/651579622 >> 172.21.15.79:0/651579622 conn(0x7f6ed8073610 msgr2=0x7f6ed806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.103+0000 7f6edd90a640 1 -- 172.21.15.79:0/651579622 shutdown_connections 2024-04-26T19:50:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.104+0000 7f6edd90a640 1 -- 172.21.15.79:0/651579622 wait complete. 2024-04-26T19:50:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6edd90a640 1 Processor -- start 2024-04-26T19:50:24.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6edd90a640 1 -- start start 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6edd90a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8144a80 0x7f6ed81441e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6edd90a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160250 0x7f6ed81f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6edd90a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ed8160630 0x7f6ed80b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6edd90a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed80727a0 con 0x7f6ed8160630 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6edd90a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8072620 con 0x7f6ed8144a80 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6edd90a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed8072920 con 0x7f6ed8160250 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6ed77fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ed8160630 0x7f6ed80b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6ed77fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ed8160630 0x7f6ed80b4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34852/0 (socket says 172.21.15.79:34852) 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.105+0000 7f6ed77fe640 1 -- 172.21.15.79:0/3949937767 learned_addr learned my addr 172.21.15.79:0/3949937767 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:24.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ecffff640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160250 0x7f6ed81f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed6ffd640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8144a80 0x7f6ed81441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed77fe640 1 -- 172.21.15.79:0/3949937767 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160250 msgr2=0x7f6ed81f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed77fe640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160250 0x7f6ed81f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed77fe640 1 -- 172.21.15.79:0/3949937767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8144a80 msgr2=0x7f6ed81441e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed77fe640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8144a80 0x7f6ed81441e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed77fe640 1 -- 172.21.15.79:0/3949937767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed815a2b0 con 0x7f6ed8160630 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed77fe640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ed8160630 0x7f6ed80b4ab0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000efc0 tx=0x7f6ed000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed4ff9640 1 -- 172.21.15.79:0/3949937767 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0018030 con 0x7f6ed8160630 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed4ff9640 1 -- 172.21.15.79:0/3949937767 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed000cdf0 con 0x7f6ed8160630 2024-04-26T19:50:24.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed815abc0 con 0x7f6ed8160630 2024-04-26T19:50:24.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6ed4ff9640 1 -- 172.21.15.79:0/3949937767 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0003bb0 con 0x7f6ed8160630 2024-04-26T19:50:24.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.106+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed815b440 con 0x7f6ed8160630 2024-04-26T19:50:24.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.107+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed815aee0 con 0x7f6ed8160630 2024-04-26T19:50:24.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.107+0000 7f6ed4ff9640 1 -- 172.21.15.79:0/3949937767 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ed0005d10 con 0x7f6ed8160630 2024-04-26T19:50:24.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.108+0000 7f6ed4ff9640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6eb806c8f0 0x7f6eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.108+0000 7f6ed4ff9640 1 -- 172.21.15.79:0/3949937767 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f6ed008c0f0 con 0x7f6ed8160630 2024-04-26T19:50:24.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.108+0000 7f6ed6ffd640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6eb806c8f0 0x7f6eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.108+0000 7f6ed6ffd640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6eb806c8f0 0x7f6eb806edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8001800 tx=0x7f6ec8001250 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:24.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.112+0000 7f6ed4ff9640 1 -- 172.21.15.79:0/3949937767 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6ed815aee0 con 0x7f6ed8160630 2024-04-26T19:50:24.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.292+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ed80d0ba0 con 0x7f6ed8160630 2024-04-26T19:50:24.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.292+0000 7f6ed4ff9640 1 -- 172.21.15.79:0/3949937767 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7f6ed0022070 con 0x7f6ed8160630 2024-04-26T19:50:24.293 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:24.294 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:20.295868+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:24.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6eb806c8f0 msgr2=0x7f6eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6eb806c8f0 0x7f6eb806edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8001800 tx=0x7f6ec8001250 comp rx=0 tx=0).stop 2024-04-26T19:50:24.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ed8160630 msgr2=0x7f6ed80b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ed8160630 0x7f6ed80b4ab0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000efc0 tx=0x7f6ed000c490 comp rx=0 tx=0).stop 2024-04-26T19:50:24.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 shutdown_connections 2024-04-26T19:50:24.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6eb806c8f0 0x7f6eb806edb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6ed8160630 0x7f6ed80b4ab0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6ed8160250 0x7f6ed81f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 --2- 172.21.15.79:0/3949937767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6ed8144a80 0x7f6ed81441e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 >> 172.21.15.79:0/3949937767 conn(0x7f6ed8073610 msgr2=0x7f6ed8162d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:24.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 shutdown_connections 2024-04-26T19:50:24.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.294+0000 7f6edd90a640 1 -- 172.21.15.79:0/3949937767 wait complete. 2024-04-26T19:50:24.306 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T19:50:24.307 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:50:24.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.405+0000 7f7e93420640 1 Processor -- start 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.405+0000 7f7e93420640 1 -- start start 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e93420640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c070160 0x7f7e8c0b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e93420640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7e8c0b1990 0x7f7e8c0af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e93420640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8c06fa10 con 0x7f7e8c0aeeb0 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e93420640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f7e8c06f7f0 con 0x7f7e8c0b1990 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e93420640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c072ea0 con 0x7f7e8c070160 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e911bc640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7e8c0aeeb0 0x7f7e8c06f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:48062/0 (socket says 172.21.15.79:48062) 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e911bc640 1 -- 172.21.15.79:0/140653120 learned_addr learned my addr 172.21.15.79:0/140653120 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e919bd640 1 --2- 172.21.15.79:0/140653120 >> v2:172.21.15.79:3300/0 conn(0x7f7e8c0b1990 0x7f7e8c0af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e83fff640 1 -- 172.21.15.79:0/140653120 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3307491133 0 0) 0x7f7e8c06fa10 con 0x7f7e8c0aeeb0 2024-04-26T19:50:24.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e83fff640 1 -- 172.21.15.79:0/140653120 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e78003520 con 0x7f7e8c0aeeb0 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e909bb640 1 --2- 172.21.15.79:0/140653120 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c070160 0x7f7e8c0b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e919bd640 1 -- 172.21.15.79:0/140653120 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c070160 msgr2=0x7f7e8c0b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e919bd640 1 --2- 172.21.15.79:0/140653120 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c070160 0x7f7e8c0b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e919bd640 1 -- 172.21.15.79:0/140653120 >> v1:172.21.15.79:6789/0 conn(0x7f7e8c0aeeb0 legacy=0x7f7e8c06f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e919bd640 1 -- 172.21.15.79:0/140653120 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c073180 con 0x7f7e8c0b1990 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.406+0000 7f7e909bb640 1 --2- 172.21.15.79:0/140653120 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c070160 0x7f7e8c0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.407+0000 7f7e919bd640 1 --2- 172.21.15.79:0/140653120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c0b1990 0x7f7e8c0af290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009d90 tx=0x7f7e8402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a64ed4806992e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.407+0000 7f7e83fff640 1 -- 172.21.15.79:0/140653120 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84031600 con 0x7f7e8c0b1990 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.407+0000 7f7e83fff640 1 -- 172.21.15.79:0/140653120 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e840317a0 con 0x7f7e8c0b1990 2024-04-26T19:50:24.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.407+0000 7f7e83fff640 1 -- 172.21.15.79:0/140653120 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84031b70 con 0x7f7e8c0b1990 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.408+0000 7f7e93420640 1 -- 172.21.15.79:0/140653120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c0b1990 msgr2=0x7f7e8c0af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.408+0000 7f7e93420640 1 --2- 172.21.15.79:0/140653120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c0b1990 0x7f7e8c0af290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009d90 tx=0x7f7e8402a5d0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.408+0000 7f7e93420640 1 -- 172.21.15.79:0/140653120 shutdown_connections 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.408+0000 7f7e93420640 1 --2- 172.21.15.79:0/140653120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c0b1990 0x7f7e8c0af290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.408+0000 7f7e93420640 1 --2- 172.21.15.79:0/140653120 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c070160 0x7f7e8c0b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.408+0000 7f7e93420640 1 -- 172.21.15.79:0/140653120 >> 172.21.15.79:0/140653120 conn(0x7f7e8c0687e0 msgr2=0x7f7e8c071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.408+0000 7f7e93420640 1 -- 172.21.15.79:0/140653120 shutdown_connections 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.408+0000 7f7e93420640 1 -- 172.21.15.79:0/140653120 wait complete. 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.409+0000 7f7e93420640 1 Processor -- start 2024-04-26T19:50:24.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.409+0000 7f7e93420640 1 -- start start 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.409+0000 7f7e93420640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c070160 0x7f7e8c160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.409+0000 7f7e93420640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e8c0aeeb0 0x7f7e8c13be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.409+0000 7f7e93420640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c0b1990 0x7f7e8c1e8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.409+0000 7f7e93420640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c149380 con 0x7f7e8c070160 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.409+0000 7f7e93420640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c073740 con 0x7f7e8c0aeeb0 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.409+0000 7f7e93420640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8c0af7d0 con 0x7f7e8c0b1990 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e911bc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c070160 0x7f7e8c160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e911bc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c070160 0x7f7e8c160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34868/0 (socket says 172.21.15.79:34868) 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e911bc640 1 -- 172.21.15.79:0/1024053695 learned_addr learned my addr 172.21.15.79:0/1024053695 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e911bc640 1 -- 172.21.15.79:0/1024053695 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c0b1990 msgr2=0x7f7e8c1e8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e911bc640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c0b1990 0x7f7e8c1e8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e911bc640 1 -- 172.21.15.79:0/1024053695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e8c0aeeb0 msgr2=0x7f7e8c13be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e911bc640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e8c0aeeb0 0x7f7e8c13be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e911bc640 1 -- 172.21.15.79:0/1024053695 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c162000 con 0x7f7e8c070160 2024-04-26T19:50:24.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e911bc640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c070160 0x7f7e8c160cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800c430 tx=0x7f7e8800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:24.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e81ffb640 1 -- 172.21.15.79:0/1024053695 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88017070 con 0x7f7e8c070160 2024-04-26T19:50:24.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e81ffb640 1 -- 172.21.15.79:0/1024053695 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e88005ce0 con 0x7f7e8c070160 2024-04-26T19:50:24.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c1622e0 con 0x7f7e8c070160 2024-04-26T19:50:24.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e81ffb640 1 -- 172.21.15.79:0/1024053695 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880072d0 con 0x7f7e8c070160 2024-04-26T19:50:24.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.410+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c1628a0 con 0x7f7e8c070160 2024-04-26T19:50:24.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.411+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8c162aa0 con 0x7f7e8c070160 2024-04-26T19:50:24.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.412+0000 7f7e81ffb640 1 -- 172.21.15.79:0/1024053695 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7e8801a070 con 0x7f7e8c070160 2024-04-26T19:50:24.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.412+0000 7f7e81ffb640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7e7006c8f0 0x7f7e7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.412+0000 7f7e909bb640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7e7006c8f0 0x7f7e7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.412+0000 7f7e81ffb640 1 -- 172.21.15.79:0/1024053695 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f7e8808abb0 con 0x7f7e8c070160 2024-04-26T19:50:24.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.412+0000 7f7e909bb640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7e7006c8f0 0x7f7e7006edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006fd0 tx=0x7f7e7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:24.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.417+0000 7f7e81ffb640 1 -- 172.21.15.79:0/1024053695 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7e8c162aa0 con 0x7f7e8c070160 2024-04-26T19:50:24.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.591+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7e8c0506c0 con 0x7f7e8c070160 2024-04-26T19:50:24.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.592+0000 7f7e81ffb640 1 -- 172.21.15.79:0/1024053695 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f7e88058ce0 con 0x7f7e8c070160 2024-04-26T19:50:24.592 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:50:24.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7e7006c8f0 msgr2=0x7f7e7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7e7006c8f0 0x7f7e7006edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c006fd0 tx=0x7f7e7c008040 comp rx=0 tx=0).stop 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c070160 msgr2=0x7f7e8c160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c070160 0x7f7e8c160cb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800c430 tx=0x7f7e8800c8f0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 shutdown_connections 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7e7006c8f0 0x7f7e7006edb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7e8c0b1990 0x7f7e8c1e8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7e8c0aeeb0 0x7f7e8c13be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 --2- 172.21.15.79:0/1024053695 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7e8c070160 0x7f7e8c160cb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 >> 172.21.15.79:0/1024053695 conn(0x7f7e8c0687e0 msgr2=0x7f7e8c0b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 shutdown_connections 2024-04-26T19:50:24.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.594+0000 7f7e93420640 1 -- 172.21.15.79:0/1024053695 wait complete. 2024-04-26T19:50:24.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:50:24.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.701+0000 7f1df2ada640 1 Processor -- start 2024-04-26T19:50:24.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.701+0000 7f1df2ada640 1 -- start start 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df2ada640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1dec06fb20 0x7f1dec0d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df2ada640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1dec148fd0 0x7f1dec0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df2ada640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dec0b1290 con 0x7f1dec06ff00 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df2ada640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f1dec0d3e60 con 0x7f1dec148fd0 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df2ada640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1dec0d4090 con 0x7f1dec06fb20 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df1077640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1dec148fd0 0x7f1dec0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df1077640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1dec148fd0 0x7f1dec0d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34882/0 (socket says 172.21.15.79:34882) 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df1077640 1 -- 172.21.15.79:0/2413760678 learned_addr learned my addr 172.21.15.79:0/2413760678 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df1077640 1 -- 172.21.15.79:0/2413760678 >> v2:172.21.15.83:3300/0 conn(0x7f1dec06fb20 msgr2=0x7f1dec0d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df1077640 1 --2- 172.21.15.79:0/2413760678 >> v2:172.21.15.83:3300/0 conn(0x7f1dec06fb20 0x7f1dec0d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df1077640 1 -- 172.21.15.79:0/2413760678 >> v1:172.21.15.83:6789/0 conn(0x7f1dec06ff00 legacy=0x7f1dec1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.702+0000 7f1df1077640 1 -- 172.21.15.79:0/2413760678 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec0d4370 con 0x7f1dec148fd0 2024-04-26T19:50:24.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.703+0000 7f1df1077640 1 --2- 172.21.15.79:0/2413760678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec148fd0 0x7f1dec0d3920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009e80 tx=0x7f1de402a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd5c868715cd9b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:24.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.703+0000 7f1deb7fe640 1 -- 172.21.15.79:0/2413760678 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de4036070 con 0x7f1dec148fd0 2024-04-26T19:50:24.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.703+0000 7f1deb7fe640 1 -- 172.21.15.79:0/2413760678 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1de4031600 con 0x7f1dec148fd0 2024-04-26T19:50:24.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.703+0000 7f1deb7fe640 1 -- 172.21.15.79:0/2413760678 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de40319d0 con 0x7f1dec148fd0 2024-04-26T19:50:24.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.704+0000 7f1df2ada640 1 -- 172.21.15.79:0/2413760678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec148fd0 msgr2=0x7f1dec0d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.704+0000 7f1df2ada640 1 --2- 172.21.15.79:0/2413760678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec148fd0 0x7f1dec0d3920 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1de4009e80 tx=0x7f1de402a7e0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.704+0000 7f1df2ada640 1 -- 172.21.15.79:0/2413760678 shutdown_connections 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.704+0000 7f1df2ada640 1 --2- 172.21.15.79:0/2413760678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec148fd0 0x7f1dec0d3920 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.704+0000 7f1df2ada640 1 --2- 172.21.15.79:0/2413760678 >> v2:172.21.15.83:3300/0 conn(0x7f1dec06fb20 0x7f1dec0d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.704+0000 7f1df2ada640 1 -- 172.21.15.79:0/2413760678 >> 172.21.15.79:0/2413760678 conn(0x7f1dec071c30 msgr2=0x7f1dec06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.704+0000 7f1df2ada640 1 -- 172.21.15.79:0/2413760678 shutdown_connections 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.704+0000 7f1df2ada640 1 -- 172.21.15.79:0/2413760678 wait complete. 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.705+0000 7f1df2ada640 1 Processor -- start 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.705+0000 7f1df2ada640 1 -- start start 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df2ada640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec06fb20 0x7f1dec15e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df2ada640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec06ff00 0x7f1dec160620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df2ada640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1dec148fd0 0x7f1dec140350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df2ada640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec0d4c10 con 0x7f1dec06ff00 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df2ada640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec0d4930 con 0x7f1dec06fb20 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df2ada640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1dec0aebb0 con 0x7f1dec148fd0 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1debfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec06ff00 0x7f1dec160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1debfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec06ff00 0x7f1dec160620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34884/0 (socket says 172.21.15.79:34884) 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1debfff640 1 -- 172.21.15.79:0/712416831 learned_addr learned my addr 172.21.15.79:0/712416831 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df0876640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec06fb20 0x7f1dec15e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df1077640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1dec148fd0 0x7f1dec140350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1debfff640 1 -- 172.21.15.79:0/712416831 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1dec148fd0 msgr2=0x7f1dec140350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1debfff640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1dec148fd0 0x7f1dec140350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1debfff640 1 -- 172.21.15.79:0/712416831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec06fb20 msgr2=0x7f1dec15e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1debfff640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec06fb20 0x7f1dec15e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1debfff640 1 -- 172.21.15.79:0/712416831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec15fcd0 con 0x7f1dec06ff00 2024-04-26T19:50:24.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df1077640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1dec148fd0 0x7f1dec140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:24.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1debfff640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec06ff00 0x7f1dec160620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800c440 tx=0x7f1dd800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:24.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1de97fa640 1 -- 172.21.15.79:0/712416831 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8017070 con 0x7f1dec06ff00 2024-04-26T19:50:24.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1de97fa640 1 -- 172.21.15.79:0/712416831 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1dd80035f0 con 0x7f1dec06ff00 2024-04-26T19:50:24.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dec15ff70 con 0x7f1dec06ff00 2024-04-26T19:50:24.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.706+0000 7f1de97fa640 1 -- 172.21.15.79:0/712416831 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd80039e0 con 0x7f1dec06ff00 2024-04-26T19:50:24.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.707+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dec15c7a0 con 0x7f1dec06ff00 2024-04-26T19:50:24.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.707+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dec0b0b50 con 0x7f1dec06ff00 2024-04-26T19:50:24.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.708+0000 7f1de97fa640 1 -- 172.21.15.79:0/712416831 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1dd801a070 con 0x7f1dec06ff00 2024-04-26T19:50:24.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.708+0000 7f1de97fa640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1dd006c8f0 0x7f1dd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:24.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.708+0000 7f1df0876640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1dd006c8f0 0x7f1dd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:24.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.708+0000 7f1de97fa640 1 -- 172.21.15.79:0/712416831 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f1dd808aa80 con 0x7f1dec06ff00 2024-04-26T19:50:24.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.709+0000 7f1df0876640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1dd006c8f0 0x7f1dd006edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009890 tx=0x7f1de0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:24.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.712+0000 7f1de97fa640 1 -- 172.21.15.79:0/712416831 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1dec0b0b50 con 0x7f1dec06ff00 2024-04-26T19:50:24.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.923+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dec0506c0 con 0x7f1dec06ff00 2024-04-26T19:50:24.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.923+0000 7f1de97fa640 1 -- 172.21.15.79:0/712416831 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7f1dd8059290 con 0x7f1dec06ff00 2024-04-26T19:50:24.924 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:24.924 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T19:50:24.925 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 2 2024-04-26T19:50:24.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1dd006c8f0 msgr2=0x7f1dd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1dd006c8f0 0x7f1dd006edb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009890 tx=0x7f1de0008040 comp rx=0 tx=0).stop 2024-04-26T19:50:24.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec06ff00 msgr2=0x7f1dec160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:24.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec06ff00 0x7f1dec160620 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800c440 tx=0x7f1dd800c900 comp rx=0 tx=0).stop 2024-04-26T19:50:24.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 shutdown_connections 2024-04-26T19:50:24.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1dd006c8f0 0x7f1dd006edb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1dec148fd0 0x7f1dec140350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1dec06ff00 0x7f1dec160620 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 --2- 172.21.15.79:0/712416831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1dec06fb20 0x7f1dec15e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:24.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 >> 172.21.15.79:0/712416831 conn(0x7f1dec071c30 msgr2=0x7f1dec144f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:24.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.925+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 shutdown_connections 2024-04-26T19:50:24.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:24.926+0000 7f1df2ada640 1 -- 172.21.15.79:0/712416831 wait complete. 2024-04-26T19:50:26.937 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:50:27.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.045+0000 7f06074d1640 1 Processor -- start 2024-04-26T19:50:27.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.045+0000 7f06074d1640 1 -- start start 2024-04-26T19:50:27.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.046+0000 7f06074d1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06000704c0 0x7f06001604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.046+0000 7f06074d1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06001634f0 con 0x7f0600163060 2024-04-26T19:50:27.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.046+0000 7f06074d1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06001637d0 con 0x7f06000708a0 2024-04-26T19:50:27.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.046+0000 7f06074d1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0600142840 con 0x7f06000704c0 2024-04-26T19:50:27.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.046+0000 7f0604a6c640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0600163060 0x7f0600142ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:48066/0 (socket says 172.21.15.79:48066) 2024-04-26T19:50:27.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f0604a6c640 1 -- 172.21.15.79:0/911705352 learned_addr learned my addr 172.21.15.79:0/911705352 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701212093 0 0) 0x7f06001634f0 con 0x7f0600163060 2024-04-26T19:50:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f060526d640 1 --2- 172.21.15.79:0/911705352 >> v2:172.21.15.83:3300/0 conn(0x7f06000704c0 0x7f06001604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f05ec003520 con 0x7f0600163060 2024-04-26T19:50:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 864180791 0 0) 0x7f05ec003520 con 0x7f0600163060 2024-04-26T19:50:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 >> v2:172.21.15.83:3300/0 conn(0x7f06000704c0 msgr2=0x7f06001604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 --2- 172.21.15.79:0/911705352 >> v2:172.21.15.83:3300/0 conn(0x7f06000704c0 0x7f06001604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 >> v1:172.21.15.83:6789/0 conn(0x7f06000708a0 legacy=0x7f06000d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0600163ab0 con 0x7f0600163060 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f05f0001670 con 0x7f0600163060 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 >> v1:172.21.15.79:6789/0 conn(0x7f0600163060 legacy=0x7f0600142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05ec0039b0 0x7f05ec003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec004800 0x7f05ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec0051c0 0x7f05ec00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0600142840 con 0x7f05ec0051c0 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec00c060 con 0x7f05ec004800 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f05ec00c260 con 0x7f05ec0039b0 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f0605a6e640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec0051c0 0x7f05ec00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f0604a6c640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec004800 0x7f05ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f060526d640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05ec0039b0 0x7f05ec003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f0604a6c640 1 -- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05ec0039b0 msgr2=0x7f05ec003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f0604a6c640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05ec0039b0 0x7f05ec003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f0604a6c640 1 -- 172.21.15.79:0/911705352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec0051c0 msgr2=0x7f05ec00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:27.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.047+0000 7f0604a6c640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec0051c0 0x7f05ec00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.048+0000 7f0604a6c640 1 -- 172.21.15.79:0/911705352 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec003520 con 0x7f05ec004800 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.048+0000 7f0605a6e640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec0051c0 0x7f05ec00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.048+0000 7f0605a6e640 1 -- 172.21.15.79:0/911705352 reap_dead start 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.048+0000 7f0604a6c640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec004800 0x7f05ec004c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c240 tx=0x7f05ec005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=798062dfd1b6f9a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.048+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0007cf0 con 0x7f05ec004800 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.048+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f00058d0 con 0x7f05ec004800 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.048+0000 7f05f7fff640 1 -- 172.21.15.79:0/911705352 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0005ca0 con 0x7f05ec004800 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.049+0000 7f06074d1640 1 -- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec004800 msgr2=0x7f05ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.049+0000 7f06074d1640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec004800 0x7f05ec004c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f05f000c240 tx=0x7f05ec005830 comp rx=0 tx=0).stop 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.049+0000 7f06074d1640 1 -- 172.21.15.79:0/911705352 shutdown_connections 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.049+0000 7f06074d1640 1 --2- 172.21.15.79:0/911705352 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05ec004800 0x7f05ec004c80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.049+0000 7f06074d1640 1 -- 172.21.15.79:0/911705352 >> 172.21.15.79:0/911705352 conn(0x7f0600073820 msgr2=0x7f06000d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.049+0000 7f06074d1640 1 -- 172.21.15.79:0/911705352 shutdown_connections 2024-04-26T19:50:27.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.049+0000 7f06074d1640 1 -- 172.21.15.79:0/911705352 wait complete. 2024-04-26T19:50:27.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.050+0000 7f06074d1640 1 Processor -- start 2024-04-26T19:50:27.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.050+0000 7f06074d1640 1 -- start start 2024-04-26T19:50:27.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f06074d1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec004800 0x7f0600159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f06074d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06000cc4b0 0x7f06000cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f06074d1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0600156b00 0x7f0600156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f06074d1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f06001604f0 con 0x7f05ec004800 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f06074d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f060006fee0 con 0x7f06000cc4b0 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f06074d1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0600160670 con 0x7f0600156b00 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f060526d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec004800 0x7f0600159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f060526d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec004800 0x7f0600159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34900/0 (socket says 172.21.15.79:34900) 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f060526d640 1 -- 172.21.15.79:0/3626762652 learned_addr learned my addr 172.21.15.79:0/3626762652 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f0604a6c640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06000cc4b0 0x7f06000cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f0605a6e640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0600156b00 0x7f0600156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f060526d640 1 -- 172.21.15.79:0/3626762652 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0600156b00 msgr2=0x7f0600156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f060526d640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0600156b00 0x7f0600156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f060526d640 1 -- 172.21.15.79:0/3626762652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06000cc4b0 msgr2=0x7f06000cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f060526d640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06000cc4b0 0x7f06000cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f060526d640 1 -- 172.21.15.79:0/3626762652 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06000ccf70 con 0x7f05ec004800 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f0604a6c640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06000cc4b0 0x7f06000cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.051+0000 7f060526d640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec004800 0x7f0600159550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f05fc011e30 tx=0x7f05fc00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.052+0000 7f05f5ffb640 1 -- 172.21.15.79:0/3626762652 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc015630 con 0x7f05ec004800 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.052+0000 7f05f5ffb640 1 -- 172.21.15.79:0/3626762652 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05fc0157d0 con 0x7f05ec004800 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.052+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06000cd250 con 0x7f05ec004800 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.052+0000 7f05f5ffb640 1 -- 172.21.15.79:0/3626762652 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc015ba0 con 0x7f05ec004800 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.052+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06000cd810 con 0x7f05ec004800 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.053+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06000506c0 con 0x7f05ec004800 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.053+0000 7f05f5ffb640 1 -- 172.21.15.79:0/3626762652 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05fc01d400 con 0x7f05ec004800 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.054+0000 7f05f5ffb640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f05e406c920 0x7f05e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.054+0000 7f05f5ffb640 1 -- 172.21.15.79:0/3626762652 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f05fc08cab0 con 0x7f05ec004800 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.054+0000 7f0604a6c640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f05e406c920 0x7f05e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.054+0000 7f0604a6c640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f05e406c920 0x7f05e406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05f0033000 tx=0x7f05f0009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.058+0000 7f05f5ffb640 1 -- 172.21.15.79:0/3626762652 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05fc019030 con 0x7f05ec004800 2024-04-26T19:50:27.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.270+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0600155f90 con 0x7f05ec004800 2024-04-26T19:50:27.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.271+0000 7f05f5ffb640 1 -- 172.21.15.79:0/3626762652 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7f05fc010040 con 0x7f05ec004800 2024-04-26T19:50:27.274 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:27.274 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4302,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T19:50:27.275 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 3 2024-04-26T19:50:27.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f05e406c920 msgr2=0x7f05e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f05e406c920 0x7f05e406ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f05f0033000 tx=0x7f05f0009030 comp rx=0 tx=0).stop 2024-04-26T19:50:27.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec004800 msgr2=0x7f0600159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec004800 0x7f0600159550 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f05fc011e30 tx=0x7f05fc00f580 comp rx=0 tx=0).stop 2024-04-26T19:50:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 shutdown_connections 2024-04-26T19:50:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f05e406c920 0x7f05e406ede0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0600156b00 0x7f0600156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06000cc4b0 0x7f06000cc930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 --2- 172.21.15.79:0/3626762652 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f05ec004800 0x7f0600159550 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 >> 172.21.15.79:0/3626762652 conn(0x7f0600073820 msgr2=0x7f06001563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 shutdown_connections 2024-04-26T19:50:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.273+0000 7f06074d1640 1 -- 172.21.15.79:0/3626762652 wait complete. 2024-04-26T19:50:27.285 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:27.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.380+0000 7fe215fe6640 1 Processor -- start 2024-04-26T19:50:27.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.380+0000 7fe215fe6640 1 -- start start 2024-04-26T19:50:27.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe215fe6640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe210160390 0x7fe2100b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe215fe6640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe210160770 0x7fe2100b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe215fe6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe2101630d0 0x7fe210163580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe215fe6640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fe210144ca0 con 0x7fe210160770 2024-04-26T19:50:27.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe215fe6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe210163e00 con 0x7fe2101630d0 2024-04-26T19:50:27.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe215fe6640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe210144f30 con 0x7fe210160390 2024-04-26T19:50:27.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20effd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe210160770 0x7fe2100b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20effd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe210160770 0x7fe2100b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34906/0 (socket says 172.21.15.79:34906) 2024-04-26T19:50:27.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20effd640 1 -- 172.21.15.79:0/407749390 learned_addr learned my addr 172.21.15.79:0/407749390 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:27.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20f7fe640 1 --2- 172.21.15.79:0/407749390 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe210160390 0x7fe2100b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20ffff640 1 --2- 172.21.15.79:0/407749390 >> v2:172.21.15.83:3300/0 conn(0x7fe2101630d0 0x7fe210163580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20effd640 1 -- 172.21.15.79:0/407749390 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe210160390 msgr2=0x7fe2100b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20effd640 1 --2- 172.21.15.79:0/407749390 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe210160390 0x7fe2100b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20effd640 1 -- 172.21.15.79:0/407749390 >> v2:172.21.15.83:3300/0 conn(0x7fe2101630d0 msgr2=0x7fe210163580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20effd640 1 --2- 172.21.15.79:0/407749390 >> v2:172.21.15.83:3300/0 conn(0x7fe2101630d0 0x7fe210163580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20effd640 1 -- 172.21.15.79:0/407749390 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe210145210 con 0x7fe210160770 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20ffff640 1 --2- 172.21.15.79:0/407749390 >> v2:172.21.15.83:3300/0 conn(0x7fe2101630d0 0x7fe210163580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.381+0000 7fe20f7fe640 1 --2- 172.21.15.79:0/407749390 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe210160390 0x7fe2100b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.382+0000 7fe20effd640 1 --2- 172.21.15.79:0/407749390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 0x7fe2100b4050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe21015d610 tx=0x7fe20402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74dacdf9bea83860 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.382+0000 7fe20e7fc640 1 -- 172.21.15.79:0/407749390 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204037070 con 0x7fe210160770 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.382+0000 7fe20e7fc640 1 -- 172.21.15.79:0/407749390 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe204032440 con 0x7fe210160770 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.382+0000 7fe20e7fc640 1 -- 172.21.15.79:0/407749390 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204032810 con 0x7fe210160770 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.383+0000 7fe215fe6640 1 -- 172.21.15.79:0/407749390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 msgr2=0x7fe2100b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.383+0000 7fe215fe6640 1 --2- 172.21.15.79:0/407749390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 0x7fe2100b4050 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe21015d610 tx=0x7fe20402a140 comp rx=0 tx=0).stop 2024-04-26T19:50:27.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.383+0000 7fe215fe6640 1 -- 172.21.15.79:0/407749390 shutdown_connections 2024-04-26T19:50:27.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.383+0000 7fe215fe6640 1 --2- 172.21.15.79:0/407749390 >> v2:172.21.15.83:3300/0 conn(0x7fe2101630d0 0x7fe210163580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.383+0000 7fe215fe6640 1 --2- 172.21.15.79:0/407749390 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 0x7fe2100b4050 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.383+0000 7fe215fe6640 1 --2- 172.21.15.79:0/407749390 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe210160390 0x7fe2100b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.383+0000 7fe215fe6640 1 -- 172.21.15.79:0/407749390 >> 172.21.15.79:0/407749390 conn(0x7fe2100b2720 msgr2=0x7fe210070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:27.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.383+0000 7fe215fe6640 1 -- 172.21.15.79:0/407749390 shutdown_connections 2024-04-26T19:50:27.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.383+0000 7fe215fe6640 1 -- 172.21.15.79:0/407749390 wait complete. 2024-04-26T19:50:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.384+0000 7fe215fe6640 1 Processor -- start 2024-04-26T19:50:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.384+0000 7fe215fe6640 1 -- start start 2024-04-26T19:50:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe215fe6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210160390 0x7fe2101589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe215fe6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 0x7fe21015dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe215fe6640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2101630d0 0x7fe21015f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe215fe6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe210145a10 con 0x7fe210160770 2024-04-26T19:50:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe215fe6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe210163ac0 con 0x7fe210160390 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe215fe6640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe210145b90 con 0x7fe2101630d0 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 0x7fe21015dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 0x7fe21015dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34914/0 (socket says 172.21.15.79:34914) 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20effd640 1 -- 172.21.15.79:0/1395273903 learned_addr learned my addr 172.21.15.79:0/1395273903 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20ffff640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2101630d0 0x7fe21015f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20effd640 1 -- 172.21.15.79:0/1395273903 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2101630d0 msgr2=0x7fe21015f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20effd640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2101630d0 0x7fe21015f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20f7fe640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210160390 0x7fe2101589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20effd640 1 -- 172.21.15.79:0/1395273903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210160390 msgr2=0x7fe2101589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20effd640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210160390 0x7fe2101589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20effd640 1 -- 172.21.15.79:0/1395273903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe21015e5b0 con 0x7fe210160770 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe20effd640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 0x7fe21015dd60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe204000c00 tx=0x7fe2040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe1f7fff640 1 -- 172.21.15.79:0/1395273903 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204037040 con 0x7fe210160770 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.385+0000 7fe1f7fff640 1 -- 172.21.15.79:0/1395273903 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe204042070 con 0x7fe210160770 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.386+0000 7fe1f7fff640 1 -- 172.21.15.79:0/1395273903 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe204030bb0 con 0x7fe210160770 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.386+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe21015edb0 con 0x7fe210160770 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.386+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe21015e2a0 con 0x7fe210160770 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.386+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2100506c0 con 0x7fe210160770 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.387+0000 7fe1f7fff640 1 -- 172.21.15.79:0/1395273903 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe204047460 con 0x7fe210160770 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.387+0000 7fe1f7fff640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe1fc06c920 0x7fe1fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.387+0000 7fe20f7fe640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe1fc06c920 0x7fe1fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.387+0000 7fe1f7fff640 1 -- 172.21.15.79:0/1395273903 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe2040ac4b0 con 0x7fe210160770 2024-04-26T19:50:27.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.388+0000 7fe20f7fe640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe1fc06c920 0x7fe1fc06ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe20000b7c0 tx=0x7fe200008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.392+0000 7fe1f7fff640 1 -- 172.21.15.79:0/1395273903 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe204045360 con 0x7fe210160770 2024-04-26T19:50:27.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.567+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe21013e450 con 0x7fe210160770 2024-04-26T19:50:27.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.568+0000 7fe1f7fff640 1 -- 172.21.15.79:0/1395273903 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12836 (secure 0 0 0) 0x7fe20407aba0 con 0x7fe210160770 2024-04-26T19:50:27.569 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:27.569 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:20.295868+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:27.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe1fc06c920 msgr2=0x7fe1fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe1fc06c920 0x7fe1fc06ede0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe20000b7c0 tx=0x7fe200008040 comp rx=0 tx=0).stop 2024-04-26T19:50:27.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 msgr2=0x7fe21015dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 0x7fe21015dd60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe204000c00 tx=0x7fe2040046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 shutdown_connections 2024-04-26T19:50:27.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe1fc06c920 0x7fe1fc06ede0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2101630d0 0x7fe21015f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe210160770 0x7fe21015dd60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 --2- 172.21.15.79:0/1395273903 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe210160390 0x7fe2101589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 >> 172.21.15.79:0/1395273903 conn(0x7fe2100b2720 msgr2=0x7fe2101442f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:27.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 shutdown_connections 2024-04-26T19:50:27.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.570+0000 7fe215fe6640 1 -- 172.21.15.79:0/1395273903 wait complete. 2024-04-26T19:50:27.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T19:50:27.582 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:50:27.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.681+0000 7ff343259640 1 Processor -- start 2024-04-26T19:50:27.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.681+0000 7ff343259640 1 -- start start 2024-04-26T19:50:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.682+0000 7ff343259640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c1598c0 0x7ff33c0b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.682+0000 7ff343259640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff33c0ba6f0 con 0x7ff33c06f210 2024-04-26T19:50:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.682+0000 7ff343259640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff33c0ba410 con 0x7ff33c160610 2024-04-26T19:50:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.682+0000 7ff343259640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7ff33c0d5100 con 0x7ff33c1598c0 2024-04-26T19:50:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.682+0000 7ff3417f6640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff33c06f210 0x7ff33c0b3920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57444/0 (socket says 172.21.15.79:57444) 2024-04-26T19:50:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.682+0000 7ff3417f6640 1 -- 172.21.15.79:0/3461559443 learned_addr learned my addr 172.21.15.79:0/3461559443 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff340ff5640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c1598c0 0x7ff33c0b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 621264261 0 0) 0x7ff33c0ba6f0 con 0x7ff33c06f210 2024-04-26T19:50:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff324003520 con 0x7ff33c06f210 2024-04-26T19:50:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1153045612 0 0) 0x7ff324003520 con 0x7ff33c06f210 2024-04-26T19:50:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c1598c0 msgr2=0x7ff33c0b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c1598c0 0x7ff33c0b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 >> v1:172.21.15.83:6789/0 conn(0x7ff33c160610 legacy=0x7ff33c0b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c0b8660 con 0x7ff33c06f210 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7ff3340019f0 con 0x7ff33c06f210 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 >> v1:172.21.15.79:6789/0 conn(0x7ff33c06f210 legacy=0x7ff33c0b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff324003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff324004390 0x7ff324004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c1598c0 0x7ff324005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c0d5100 con 0x7ff3240039e0 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff32400ba70 con 0x7ff324004390 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7ff32400bc70 con 0x7ff33c1598c0 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff340ff5640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff324003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff33bfff640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff324004390 0x7ff324004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff3417f6640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c1598c0 0x7ff324005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff340ff5640 1 -- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c1598c0 msgr2=0x7ff324005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff340ff5640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c1598c0 0x7ff324005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff340ff5640 1 -- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff324004390 msgr2=0x7ff324004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:27.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff340ff5640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff324004390 0x7ff324004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.683+0000 7ff340ff5640 1 -- 172.21.15.79:0/3461559443 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff324003520 con 0x7ff3240039e0 2024-04-26T19:50:27.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.684+0000 7ff340ff5640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff324003dc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009ac0 tx=0x7ff324005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efff61510ea9494 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.684+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c036070 con 0x7ff3240039e0 2024-04-26T19:50:27.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.684+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32c031400 con 0x7ff3240039e0 2024-04-26T19:50:27.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.684+0000 7ff33b7fe640 1 -- 172.21.15.79:0/3461559443 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c009690 con 0x7ff3240039e0 2024-04-26T19:50:27.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.685+0000 7ff343259640 1 -- 172.21.15.79:0/3461559443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 msgr2=0x7ff324003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.685+0000 7ff343259640 1 --2- 172.21.15.79:0/3461559443 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff324003dc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff32c009ac0 tx=0x7ff324005830 comp rx=0 tx=0).stop 2024-04-26T19:50:27.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.685+0000 7ff3417f6640 1 -- 172.21.15.79:0/3461559443 reap_dead start 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.685+0000 7ff343259640 1 -- 172.21.15.79:0/3461559443 shutdown_connections 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.685+0000 7ff343259640 1 -- 172.21.15.79:0/3461559443 >> 172.21.15.79:0/3461559443 conn(0x7ff33c0b4930 msgr2=0x7ff33c0b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.685+0000 7ff343259640 1 -- 172.21.15.79:0/3461559443 shutdown_connections 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.685+0000 7ff343259640 1 -- 172.21.15.79:0/3461559443 wait complete. 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.686+0000 7ff343259640 1 Processor -- start 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.686+0000 7ff343259640 1 -- start start 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff343259640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff33c1580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff343259640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c15e710 0x7ff33c15eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff343259640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff33c1f8320 0x7ff33c1f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff343259640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c159e20 con 0x7ff3240039e0 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff343259640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c159ca0 con 0x7ff33c1f8320 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff343259640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7ff33c0b9140 con 0x7ff33c15e710 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff340ff5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff33c1580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff340ff5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff33c1580d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52932/0 (socket says 172.21.15.79:52932) 2024-04-26T19:50:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff340ff5640 1 -- 172.21.15.79:0/2010125227 learned_addr learned my addr 172.21.15.79:0/2010125227 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff340ff5640 1 -- 172.21.15.79:0/2010125227 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c15e710 msgr2=0x7ff33c15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff340ff5640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c15e710 0x7ff33c15eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff340ff5640 1 -- 172.21.15.79:0/2010125227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff33c1f8320 msgr2=0x7ff33c1f87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff3417f6640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff33c1f8320 0x7ff33c1f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff340ff5640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff33c1f8320 0x7ff33c1f87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff340ff5640 1 -- 172.21.15.79:0/2010125227 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c1f90a0 con 0x7ff3240039e0 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff3417f6640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff33c1f8320 0x7ff33c1f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff340ff5640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff33c1580d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff32c034040 tx=0x7ff32c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff3397fa640 1 -- 172.21.15.79:0/2010125227 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c040020 con 0x7ff3240039e0 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff3397fa640 1 -- 172.21.15.79:0/2010125227 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff32c003bc0 con 0x7ff3240039e0 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff3397fa640 1 -- 172.21.15.79:0/2010125227 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff32c036040 con 0x7ff3240039e0 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c147870 con 0x7ff3240039e0 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.687+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c147e30 con 0x7ff3240039e0 2024-04-26T19:50:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.689+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff33c0506c0 con 0x7ff3240039e0 2024-04-26T19:50:27.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.689+0000 7ff3397fa640 1 -- 172.21.15.79:0/2010125227 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff32c043020 con 0x7ff3240039e0 2024-04-26T19:50:27.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.689+0000 7ff3397fa640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff32806c920 0x7ff32806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.689+0000 7ff33bfff640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff32806c920 0x7ff32806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.689+0000 7ff3397fa640 1 -- 172.21.15.79:0/2010125227 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7ff32c0ab880 con 0x7ff3240039e0 2024-04-26T19:50:27.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.690+0000 7ff33bfff640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff32806c920 0x7ff32806ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff330007fd0 tx=0x7ff330009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.694+0000 7ff3397fa640 1 -- 172.21.15.79:0/2010125227 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff32c04a070 con 0x7ff3240039e0 2024-04-26T19:50:27.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.865+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff33c0b3b70 con 0x7ff3240039e0 2024-04-26T19:50:27.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.866+0000 7ff3397fa640 1 -- 172.21.15.79:0/2010125227 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7ff32c0044e0 con 0x7ff3240039e0 2024-04-26T19:50:27.867 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:50:27.868 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff32806c920 msgr2=0x7ff32806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff32806c920 0x7ff32806ede0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff330007fd0 tx=0x7ff330009040 comp rx=0 tx=0).stop 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 msgr2=0x7ff33c1580d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff33c1580d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff32c034040 tx=0x7ff32c004730 comp rx=0 tx=0).stop 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 shutdown_connections 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff32806c920 0x7ff32806ede0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff33c1f8320 0x7ff33c1f87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff33c15e710 0x7ff33c15eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 --2- 172.21.15.79:0/2010125227 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3240039e0 0x7ff33c1580d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 >> 172.21.15.79:0/2010125227 conn(0x7ff33c0b4930 msgr2=0x7ff33c0d4590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 shutdown_connections 2024-04-26T19:50:27.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.868+0000 7ff343259640 1 -- 172.21.15.79:0/2010125227 wait complete. 2024-04-26T19:50:27.879 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T19:50:27.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.976+0000 7f9da4d9d640 1 Processor -- start 2024-04-26T19:50:27.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.976+0000 7f9da4d9d640 1 -- start start 2024-04-26T19:50:27.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9da4d9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9da0156670 0x7f9da015d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9da4d9d640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da015d670 con 0x7f9da00cd9c0 2024-04-26T19:50:27.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9da4d9d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9da00ce1a0 con 0x7f9da00b44f0 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9da4d9d640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f9da0073110 con 0x7f9da0156670 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9d9e59c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9da0156670 0x7f9da015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9d9dd9b640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9da00cd9c0 0x7f9da00b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57460/0 (socket says 172.21.15.79:57460) 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9d9dd9b640 1 -- 172.21.15.79:0/2605557313 learned_addr learned my addr 172.21.15.79:0/2605557313 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9d9e59c640 1 -- 172.21.15.79:0/2605557313 >> v1:172.21.15.83:6789/0 conn(0x7f9da00b44f0 legacy=0x7f9da00cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9d9e59c640 1 -- 172.21.15.79:0/2605557313 >> v1:172.21.15.79:6789/0 conn(0x7f9da00cd9c0 legacy=0x7f9da00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9d9e59c640 1 -- 172.21.15.79:0/2605557313 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da00cdec0 con 0x7f9da0156670 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.977+0000 7f9d9d59a640 1 -- 172.21.15.79:0/2605557313 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2238748644 0 0) 0x7f9d90001110 con 0x7f9da00cd9c0 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9d9e59c640 1 --2- 172.21.15.79:0/2605557313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 0x7f9da015d130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9da0073290 tx=0x7f9d9402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b028c411cb8470 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9d9d59a640 1 -- 172.21.15.79:0/2605557313 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94037070 con 0x7f9da0156670 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9d9d59a640 1 -- 172.21.15.79:0/2605557313 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94032440 con 0x7f9da0156670 2024-04-26T19:50:27.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9d9d59a640 1 -- 172.21.15.79:0/2605557313 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94032810 con 0x7f9da0156670 2024-04-26T19:50:27.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9da4d9d640 1 -- 172.21.15.79:0/2605557313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 msgr2=0x7f9da015d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9da4d9d640 1 --2- 172.21.15.79:0/2605557313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 0x7f9da015d130 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9da0073290 tx=0x7f9d9402a210 comp rx=0 tx=0).stop 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9da4d9d640 1 -- 172.21.15.79:0/2605557313 shutdown_connections 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9da4d9d640 1 --2- 172.21.15.79:0/2605557313 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 0x7f9da015d130 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9da4d9d640 1 -- 172.21.15.79:0/2605557313 >> 172.21.15.79:0/2605557313 conn(0x7f9da00b39d0 msgr2=0x7f9da006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9da4d9d640 1 -- 172.21.15.79:0/2605557313 shutdown_connections 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.978+0000 7f9da4d9d640 1 -- 172.21.15.79:0/2605557313 wait complete. 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.980+0000 7f9da4d9d640 1 Processor -- start 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.980+0000 7f9da4d9d640 1 -- start start 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.980+0000 7f9da4d9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da00b44f0 0x7f9da01f5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.980+0000 7f9da4d9d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9da00cd9c0 0x7f9da0153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.980+0000 7f9da4d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 0x7f9da0149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.980+0000 7f9da4d9d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0156a50 con 0x7f9da0156670 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.980+0000 7f9da4d9d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9da0072da0 con 0x7f9da00b44f0 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.980+0000 7f9da4d9d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9da0156bd0 con 0x7f9da00cd9c0 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9ed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 0x7f9da0149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9ed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 0x7f9da0149b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52952/0 (socket says 172.21.15.79:52952) 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9ed9d640 1 -- 172.21.15.79:0/3121062777 learned_addr learned my addr 172.21.15.79:0/3121062777 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9e59c640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da00b44f0 0x7f9da01f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9dd9b640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9da00cd9c0 0x7f9da0153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9ed9d640 1 -- 172.21.15.79:0/3121062777 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9da00cd9c0 msgr2=0x7f9da0153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9ed9d640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9da00cd9c0 0x7f9da0153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9ed9d640 1 -- 172.21.15.79:0/3121062777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da00b44f0 msgr2=0x7f9da01f5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:27.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9ed9d640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da00b44f0 0x7f9da01f5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9ed9d640 1 -- 172.21.15.79:0/3121062777 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da01f5b90 con 0x7f9da0156670 2024-04-26T19:50:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d9ed9d640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 0x7f9da0149b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800c760 tx=0x7f9d9800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d8effd640 1 -- 172.21.15.79:0/3121062777 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98003b70 con 0x7f9da0156670 2024-04-26T19:50:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d8effd640 1 -- 172.21.15.79:0/3121062777 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d98003d10 con 0x7f9da0156670 2024-04-26T19:50:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da015a890 con 0x7f9da0156670 2024-04-26T19:50:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9d8effd640 1 -- 172.21.15.79:0/3121062777 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9800aa30 con 0x7f9da0156670 2024-04-26T19:50:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.981+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da015ae50 con 0x7f9da0156670 2024-04-26T19:50:27.985 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.982+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9da00506c0 con 0x7f9da0156670 2024-04-26T19:50:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.983+0000 7f9d8effd640 1 -- 172.21.15.79:0/3121062777 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d9800abd0 con 0x7f9da0156670 2024-04-26T19:50:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.983+0000 7f9d8effd640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9d8006c920 0x7f9d8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.983+0000 7f9d8effd640 1 -- 172.21.15.79:0/3121062777 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f9d9808b610 con 0x7f9da0156670 2024-04-26T19:50:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.983+0000 7f9d9e59c640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9d8006c920 0x7f9d8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:27.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.983+0000 7f9d9e59c640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9d8006c920 0x7f9d8006ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d94034000 tx=0x7f9d9402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:27.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:27.987+0000 7f9d8effd640 1 -- 172.21.15.79:0/3121062777 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9da01f5b90 con 0x7f9da0156670 2024-04-26T19:50:28.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.166+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9da0069bc0 con 0x7f9da0156670 2024-04-26T19:50:28.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.166+0000 7f9d8effd640 1 -- 172.21.15.79:0/3121062777 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f9d98059e20 con 0x7f9da0156670 2024-04-26T19:50:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:28.167 INFO:teuthology.orchestra.run.smithi079.stdout:[ 2024-04-26T19:50:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:50:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 1, 2024-04-26T19:50:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": ".mgr" 2024-04-26T19:50:28.167 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:50:28.167 INFO:teuthology.orchestra.run.smithi079.stdout:] 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9d8006c920 msgr2=0x7f9d8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9d8006c920 0x7f9d8006ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9d94034000 tx=0x7f9d9402e000 comp rx=0 tx=0).stop 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 msgr2=0x7f9da0149b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 0x7f9da0149b50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800c760 tx=0x7f9d9800cc20 comp rx=0 tx=0).stop 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 shutdown_connections 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9d8006c920 0x7f9d8006ede0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9da0156670 0x7f9da0149b50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9da00cd9c0 0x7f9da0153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 --2- 172.21.15.79:0/3121062777 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9da00b44f0 0x7f9da01f5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 >> 172.21.15.79:0/3121062777 conn(0x7f9da00b39d0 msgr2=0x7f9da0155920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 shutdown_connections 2024-04-26T19:50:28.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.168+0000 7f9da4d9d640 1 -- 172.21.15.79:0/3121062777 wait complete. 2024-04-26T19:50:28.179 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T19:50:28.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T19:50:28.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.280+0000 7f574e346640 1 Processor -- start 2024-04-26T19:50:28.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.280+0000 7f574e346640 1 -- start start 2024-04-26T19:50:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f574e346640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5748159b30 0x7f574806f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f574e346640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574806e480 con 0x7f574806df80 2024-04-26T19:50:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f574e346640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f574806e760 con 0x7f5748144ca0 2024-04-26T19:50:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f574e346640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5748145bb0 con 0x7f5748159b30 2024-04-26T19:50:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f57477fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f574806df80 0x7f5748160560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57472/0 (socket says 172.21.15.79:57472) 2024-04-26T19:50:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f57477fe640 1 -- 172.21.15.79:0/4205950995 learned_addr learned my addr 172.21.15.79:0/4205950995 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5747fff640 1 --2- 172.21.15.79:0/4205950995 >> v2:172.21.15.83:3300/0 conn(0x7f5748159b30 0x7f574806f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3116704587 0 0) 0x7f574806e480 con 0x7f574806df80 2024-04-26T19:50:28.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5730003520 con 0x7f574806df80 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3411713977 0 0) 0x7f5730003520 con 0x7f574806df80 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 >> v2:172.21.15.83:3300/0 conn(0x7f5748159b30 msgr2=0x7f574806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 --2- 172.21.15.79:0/4205950995 >> v2:172.21.15.83:3300/0 conn(0x7f5748159b30 0x7f574806f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 >> v1:172.21.15.83:6789/0 conn(0x7f5748144ca0 legacy=0x7f57481450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f574806ea40 con 0x7f574806df80 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f5738001670 con 0x7f574806df80 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 >> v1:172.21.15.79:6789/0 conn(0x7f574806df80 legacy=0x7f5748160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57300039b0 0x7f5730003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5730004800 0x7f5730004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300051c0 0x7f573000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f573000c030 con 0x7f57300039b0 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f573000c1e0 con 0x7f57300051c0 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f573000c3e0 con 0x7f5730004800 2024-04-26T19:50:28.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.281+0000 7f5747fff640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57300039b0 0x7f5730003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f57477fe640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5730004800 0x7f5730004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f574c8e3640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300051c0 0x7f573000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f5747fff640 1 -- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5730004800 msgr2=0x7f5730004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f5747fff640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5730004800 0x7f5730004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f5747fff640 1 -- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300051c0 msgr2=0x7f573000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f5747fff640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300051c0 0x7f573000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f5747fff640 1 -- 172.21.15.79:0/4205950995 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5730003520 con 0x7f57300039b0 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f57477fe640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5730004800 0x7f5730004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f574c8e3640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300051c0 0x7f573000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f574c8e3640 1 -- 172.21.15.79:0/4205950995 reap_dead start 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f5747fff640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57300039b0 0x7f5730003dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f573c009ac0 tx=0x7f5730005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286da52bcb37add7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c036070 con 0x7f57300039b0 2024-04-26T19:50:28.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.282+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f573c031400 con 0x7f57300039b0 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.283+0000 7f5746ffd640 1 -- 172.21.15.79:0/4205950995 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c009690 con 0x7f57300039b0 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.283+0000 7f574e346640 1 -- 172.21.15.79:0/4205950995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57300039b0 msgr2=0x7f5730003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.283+0000 7f574e346640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57300039b0 0x7f5730003dd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f573c009ac0 tx=0x7f5730005830 comp rx=0 tx=0).stop 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.283+0000 7f574e346640 1 -- 172.21.15.79:0/4205950995 shutdown_connections 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.283+0000 7f574e346640 1 --2- 172.21.15.79:0/4205950995 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57300039b0 0x7f5730003dd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.283+0000 7f574e346640 1 -- 172.21.15.79:0/4205950995 >> 172.21.15.79:0/4205950995 conn(0x7f57480685d0 msgr2=0x7f57480d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.283+0000 7f574e346640 1 -- 172.21.15.79:0/4205950995 shutdown_connections 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.283+0000 7f574e346640 1 -- 172.21.15.79:0/4205950995 wait complete. 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.284+0000 7f574e346640 1 Processor -- start 2024-04-26T19:50:28.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.284+0000 7f574e346640 1 -- start start 2024-04-26T19:50:28.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f574e346640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300039b0 0x7f57480d4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:28.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f574e346640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5748145830 0x7f574813fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:28.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f574e346640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5748145c90 0x7f57481402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:28.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f574e346640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5748159f10 con 0x7f5748145c90 2024-04-26T19:50:28.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f574e346640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f574806f090 con 0x7f57300039b0 2024-04-26T19:50:28.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f574e346640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f574806f520 con 0x7f5748145830 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f574c8e3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5748145c90 0x7f57481402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f574c8e3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5748145c90 0x7f57481402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52966/0 (socket says 172.21.15.79:52966) 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f574c8e3640 1 -- 172.21.15.79:0/3631554402 learned_addr learned my addr 172.21.15.79:0/3631554402 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f5747fff640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300039b0 0x7f57480d4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.285+0000 7f57477fe640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5748145830 0x7f574813fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f574c8e3640 1 -- 172.21.15.79:0/3631554402 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5748145830 msgr2=0x7f574813fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f574c8e3640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5748145830 0x7f574813fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f574c8e3640 1 -- 172.21.15.79:0/3631554402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300039b0 msgr2=0x7f57480d4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f574c8e3640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300039b0 0x7f57480d4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f574c8e3640 1 -- 172.21.15.79:0/3631554402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5748146260 con 0x7f5748145c90 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f574c8e3640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5748145c90 0x7f57481402c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5748163000 tx=0x7f574000e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:28.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f5744ff9640 1 -- 172.21.15.79:0/3631554402 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5740003b80 con 0x7f5748145c90 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f5744ff9640 1 -- 172.21.15.79:0/3631554402 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5740018070 con 0x7f5748145c90 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5748146540 con 0x7f5748145c90 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f5744ff9640 1 -- 172.21.15.79:0/3631554402 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57400050d0 con 0x7f5748145c90 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.286+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5748146c00 con 0x7f5748145c90 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.287+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57480506c0 con 0x7f5748145c90 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.287+0000 7f5744ff9640 1 -- 172.21.15.79:0/3631554402 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5740005270 con 0x7f5748145c90 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.288+0000 7f5744ff9640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f573406c920 0x7f573406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.288+0000 7f5744ff9640 1 -- 172.21.15.79:0/3631554402 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f574008b3d0 con 0x7f5748145c90 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.288+0000 7f5747fff640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f573406c920 0x7f573406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:28.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.288+0000 7f5747fff640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f573406c920 0x7f573406ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f573c033000 tx=0x7f573c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:28.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.292+0000 7f5744ff9640 1 -- 172.21.15.79:0/3631554402 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5740059be0 con 0x7f5748145c90 2024-04-26T19:50:28.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:28.465+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f57480740a0 con 0x7f5748145c90 2024-04-26T19:50:29.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.078+0000 7f5744ff9640 1 -- 172.21.15.79:0/3631554402 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f57400594e0 con 0x7f5748145c90 2024-04-26T19:50:29.079 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' created 2024-04-26T19:50:29.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f573406c920 msgr2=0x7f573406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:29.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f573406c920 0x7f573406ede0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f573c033000 tx=0x7f573c008030 comp rx=0 tx=0).stop 2024-04-26T19:50:29.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5748145c90 msgr2=0x7f57481402c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:29.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5748145c90 0x7f57481402c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5748163000 tx=0x7f574000e960 comp rx=0 tx=0).stop 2024-04-26T19:50:29.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 shutdown_connections 2024-04-26T19:50:29.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f573406c920 0x7f573406ede0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5748145c90 0x7f57481402c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5748145830 0x7f574813fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 --2- 172.21.15.79:0/3631554402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57300039b0 0x7f57480d4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 >> 172.21.15.79:0/3631554402 conn(0x7f57480685d0 msgr2=0x7f5748141260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:29.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 shutdown_connections 2024-04-26T19:50:29.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.081+0000 7f574e346640 1 -- 172.21.15.79:0/3631554402 wait complete. 2024-04-26T19:50:29.099 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-26T19:50:29.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.206+0000 7f0b2ab6a640 1 Processor -- start 2024-04-26T19:50:29.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b2ab6a640 1 -- start start 2024-04-26T19:50:29.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b2ab6a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0b2406fb20 0x7f0b240d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:29.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b2ab6a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406ff00 0x7f0b240d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:29.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b2ab6a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0b241488c0 0x7f0b24148d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:29.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b2ab6a640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f0b240d3e60 con 0x7f0b2406fb20 2024-04-26T19:50:29.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b2ab6a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0b240b1290 con 0x7f0b241488c0 2024-04-26T19:50:29.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b2ab6a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0b240d4090 con 0x7f0b2406ff00 2024-04-26T19:50:29.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b28906640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0b2406fb20 0x7f0b240d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:29.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b28906640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0b2406fb20 0x7f0b240d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52974/0 (socket says 172.21.15.79:52974) 2024-04-26T19:50:29.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.207+0000 7f0b28906640 1 -- 172.21.15.79:0/1806367096 learned_addr learned my addr 172.21.15.79:0/1806367096 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b23fff640 1 --2- 172.21.15.79:0/1806367096 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406ff00 0x7f0b240d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b29107640 1 --2- 172.21.15.79:0/1806367096 >> v2:172.21.15.83:3300/0 conn(0x7f0b241488c0 0x7f0b24148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b28906640 1 -- 172.21.15.79:0/1806367096 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406ff00 msgr2=0x7f0b240d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b28906640 1 --2- 172.21.15.79:0/1806367096 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406ff00 0x7f0b240d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b28906640 1 -- 172.21.15.79:0/1806367096 >> v2:172.21.15.83:3300/0 conn(0x7f0b241488c0 msgr2=0x7f0b24148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b28906640 1 --2- 172.21.15.79:0/1806367096 >> v2:172.21.15.83:3300/0 conn(0x7f0b241488c0 0x7f0b24148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b28906640 1 -- 172.21.15.79:0/1806367096 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b240d4370 con 0x7f0b2406fb20 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b23fff640 1 --2- 172.21.15.79:0/1806367096 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406ff00 0x7f0b240d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b29107640 1 --2- 172.21.15.79:0/1806367096 >> v2:172.21.15.83:3300/0 conn(0x7f0b241488c0 0x7f0b24148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b28906640 1 --2- 172.21.15.79:0/1806367096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2406fb20 0x7f0b240d2cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b240b11e0 tx=0x7f0b1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eaf76a3c86ff130 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.208+0000 7f0b237fe640 1 -- 172.21.15.79:0/1806367096 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18037070 con 0x7f0b2406fb20 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b237fe640 1 -- 172.21.15.79:0/1806367096 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b18032440 con 0x7f0b2406fb20 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b237fe640 1 -- 172.21.15.79:0/1806367096 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18032810 con 0x7f0b2406fb20 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/1806367096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2406fb20 msgr2=0x7f0b240d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/1806367096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2406fb20 0x7f0b240d2cd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0b240b11e0 tx=0x7f0b1802a270 comp rx=0 tx=0).stop 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/1806367096 shutdown_connections 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/1806367096 >> v2:172.21.15.83:3300/0 conn(0x7f0b241488c0 0x7f0b24148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/1806367096 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406ff00 0x7f0b240d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/1806367096 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b2406fb20 0x7f0b240d2cd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/1806367096 >> 172.21.15.79:0/1806367096 conn(0x7f0b24071c30 msgr2=0x7f0b2406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/1806367096 shutdown_connections 2024-04-26T19:50:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.209+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/1806367096 wait complete. 2024-04-26T19:50:29.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.210+0000 7f0b2ab6a640 1 Processor -- start 2024-04-26T19:50:29.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.210+0000 7f0b2ab6a640 1 -- start start 2024-04-26T19:50:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b2ab6a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406fb20 0x7f0b2413f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b2ab6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2406ff00 0x7f0b2413d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b2ab6a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b241488c0 0x7f0b24142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b2ab6a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240d4c10 con 0x7f0b241488c0 2024-04-26T19:50:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b2ab6a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0b240d4930 con 0x7f0b2406ff00 2024-04-26T19:50:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b2ab6a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0b24149270 con 0x7f0b2406fb20 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b29107640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b241488c0 0x7f0b24142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b29107640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b241488c0 0x7f0b24142840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52978/0 (socket says 172.21.15.79:52978) 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b29107640 1 -- 172.21.15.79:0/90096791 learned_addr learned my addr 172.21.15.79:0/90096791 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b28906640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406fb20 0x7f0b2413f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.211+0000 7f0b23fff640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2406ff00 0x7f0b2413d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b29107640 1 -- 172.21.15.79:0/90096791 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406fb20 msgr2=0x7f0b2413f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b29107640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406fb20 0x7f0b2413f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b29107640 1 -- 172.21.15.79:0/90096791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2406ff00 msgr2=0x7f0b2413d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b29107640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2406ff00 0x7f0b2413d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b29107640 1 -- 172.21.15.79:0/90096791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2413f7e0 con 0x7f0b241488c0 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b29107640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b241488c0 0x7f0b24142840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00cef0 tx=0x7f0b1c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b217fa640 1 -- 172.21.15.79:0/90096791 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c01a070 con 0x7f0b241488c0 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b217fa640 1 -- 172.21.15.79:0/90096791 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b1c00d860 con 0x7f0b241488c0 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b2413fac0 con 0x7f0b241488c0 2024-04-26T19:50:29.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b217fa640 1 -- 172.21.15.79:0/90096791 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c00dc50 con 0x7f0b241488c0 2024-04-26T19:50:29.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.212+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b24140080 con 0x7f0b241488c0 2024-04-26T19:50:29.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.213+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b240506c0 con 0x7f0b241488c0 2024-04-26T19:50:29.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.213+0000 7f0b217fa640 1 -- 172.21.15.79:0/90096791 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b1c01d070 con 0x7f0b241488c0 2024-04-26T19:50:29.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.214+0000 7f0b217fa640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0b0806c8f0 0x7f0b0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:29.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.214+0000 7f0b217fa640 1 -- 172.21.15.79:0/90096791 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f0b1c08c190 con 0x7f0b241488c0 2024-04-26T19:50:29.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.214+0000 7f0b28906640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0b0806c8f0 0x7f0b0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:29.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.214+0000 7f0b28906640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0b0806c8f0 0x7f0b0806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:29.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.218+0000 7f0b217fa640 1 -- 172.21.15.79:0/90096791 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0b1c05a210 con 0x7f0b241488c0 2024-04-26T19:50:29.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:29.394+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f0b240d3700 con 0x7f0b241488c0 2024-04-26T19:50:30.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.088+0000 7f0b217fa640 1 -- 172.21.15.79:0/90096791 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f0b1c05e510 con 0x7f0b241488c0 2024-04-26T19:50:30.089 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' created 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0b0806c8f0 msgr2=0x7f0b0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0b0806c8f0 0x7f0b0806edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0b18034000 tx=0x7f0b1802e000 comp rx=0 tx=0).stop 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b241488c0 msgr2=0x7f0b24142840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b241488c0 0x7f0b24142840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00cef0 tx=0x7f0b1c00a710 comp rx=0 tx=0).stop 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 shutdown_connections 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0b0806c8f0 0x7f0b0806edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0b241488c0 0x7f0b24142840 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0b2406ff00 0x7f0b2413d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 --2- 172.21.15.79:0/90096791 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0b2406fb20 0x7f0b2413f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 >> 172.21.15.79:0/90096791 conn(0x7f0b24071c30 msgr2=0x7f0b24162710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:30.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 shutdown_connections 2024-04-26T19:50:30.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.091+0000 7f0b2ab6a640 1 -- 172.21.15.79:0/90096791 wait complete. 2024-04-26T19:50:30.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T19:50:30.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.220+0000 7f72accda640 1 Processor -- start 2024-04-26T19:50:30.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.220+0000 7f72accda640 1 -- start start 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.221+0000 7f72accda640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f72a815d010 0x7f72a8071e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.221+0000 7f72accda640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a8072510 con 0x7f72a80d1730 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.221+0000 7f72accda640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72a80727f0 con 0x7f72a806f0e0 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.221+0000 7f72accda640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f72a8073120 con 0x7f72a815d010 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.221+0000 7f72a659c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f72a815d010 0x7f72a8071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.221+0000 7f72a6d9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f72a80d1730 0x7f72a80735b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57486/0 (socket says 172.21.15.79:57486) 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.221+0000 7f72a6d9d640 1 -- 172.21.15.79:0/1116993400 learned_addr learned my addr 172.21.15.79:0/1116993400 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.222+0000 7f72a659c640 1 -- 172.21.15.79:0/1116993400 >> v1:172.21.15.83:6789/0 conn(0x7f72a806f0e0 legacy=0x7f72a80d1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.222+0000 7f72a659c640 1 -- 172.21.15.79:0/1116993400 >> v1:172.21.15.79:6789/0 conn(0x7f72a80d1730 legacy=0x7f72a80735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.222+0000 7f72a659c640 1 -- 172.21.15.79:0/1116993400 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a8072ad0 con 0x7f72a815d010 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.222+0000 7f72a559a640 1 -- 172.21.15.79:0/1116993400 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1079616346 0 0) 0x7f72a8072510 con 0x7f72a80d1730 2024-04-26T19:50:30.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.222+0000 7f72a659c640 1 --2- 172.21.15.79:0/1116993400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 0x7f72a8071e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72a00097a0 tx=0x7f72a002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6353d9b01ea3ab01 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:30.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.222+0000 7f72a559a640 1 -- 172.21.15.79:0/1116993400 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0037070 con 0x7f72a815d010 2024-04-26T19:50:30.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.222+0000 7f72a559a640 1 -- 172.21.15.79:0/1116993400 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72a0032440 con 0x7f72a815d010 2024-04-26T19:50:30.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.222+0000 7f72a559a640 1 -- 172.21.15.79:0/1116993400 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0032810 con 0x7f72a815d010 2024-04-26T19:50:30.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.223+0000 7f72accda640 1 -- 172.21.15.79:0/1116993400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 msgr2=0x7f72a8071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:30.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.223+0000 7f72accda640 1 --2- 172.21.15.79:0/1116993400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 0x7f72a8071e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f72a00097a0 tx=0x7f72a002a230 comp rx=0 tx=0).stop 2024-04-26T19:50:30.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.223+0000 7f72accda640 1 -- 172.21.15.79:0/1116993400 shutdown_connections 2024-04-26T19:50:30.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.223+0000 7f72accda640 1 --2- 172.21.15.79:0/1116993400 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 0x7f72a8071e10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:30.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.223+0000 7f72accda640 1 -- 172.21.15.79:0/1116993400 >> 172.21.15.79:0/1116993400 conn(0x7f72a8068ad0 msgr2=0x7f72a80b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:30.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.223+0000 7f72accda640 1 -- 172.21.15.79:0/1116993400 shutdown_connections 2024-04-26T19:50:30.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.223+0000 7f72accda640 1 -- 172.21.15.79:0/1116993400 wait complete. 2024-04-26T19:50:30.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.224+0000 7f72accda640 1 Processor -- start 2024-04-26T19:50:30.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.224+0000 7f72accda640 1 -- start start 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72accda640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a806f0e0 0x7f72a8159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72accda640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f72a80d1730 0x7f72a81f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72accda640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 0x7f72a814ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72accda640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f72a815d3f0 con 0x7f72a815d010 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72accda640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f72a80d2120 con 0x7f72a806f0e0 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72accda640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f72a815d570 con 0x7f72a80d1730 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72a6d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 0x7f72a814ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72a6d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 0x7f72a814ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:52994/0 (socket says 172.21.15.79:52994) 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72a6d9d640 1 -- 172.21.15.79:0/158605497 learned_addr learned my addr 172.21.15.79:0/158605497 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72a659c640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a806f0e0 0x7f72a8159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.225+0000 7f72a5d9b640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f72a80d1730 0x7f72a81f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:30.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f72a6d9d640 1 -- 172.21.15.79:0/158605497 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f72a80d1730 msgr2=0x7f72a81f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f72a6d9d640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f72a80d1730 0x7f72a81f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f72a6d9d640 1 -- 172.21.15.79:0/158605497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a806f0e0 msgr2=0x7f72a8159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f72a6d9d640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a806f0e0 0x7f72a8159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f72a6d9d640 1 -- 172.21.15.79:0/158605497 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a8155580 con 0x7f72a815d010 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f72a6d9d640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 0x7f72a814ec50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f729800c4b0 tx=0x7f729800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f7296ffd640 1 -- 172.21.15.79:0/158605497 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298018070 con 0x7f72a815d010 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f7296ffd640 1 -- 172.21.15.79:0/158605497 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72980074e0 con 0x7f72a815d010 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a815a230 con 0x7f72a815d010 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f7296ffd640 1 -- 172.21.15.79:0/158605497 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298004a20 con 0x7f72a815d010 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.226+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a815a7f0 con 0x7f72a815d010 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.227+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a80506c0 con 0x7f72a815d010 2024-04-26T19:50:30.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.228+0000 7f7296ffd640 1 -- 172.21.15.79:0/158605497 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f729801b070 con 0x7f72a815d010 2024-04-26T19:50:30.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.228+0000 7f7296ffd640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f728806c8f0 0x7f728806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:30.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.228+0000 7f7296ffd640 1 -- 172.21.15.79:0/158605497 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7f729808abd0 con 0x7f72a815d010 2024-04-26T19:50:30.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.228+0000 7f72a659c640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f728806c8f0 0x7f728806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:30.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.228+0000 7f72a659c640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f728806c8f0 0x7f728806edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72a0034000 tx=0x7f72a002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:30.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.233+0000 7f7296ffd640 1 -- 172.21.15.79:0/158605497 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f72a8155580 con 0x7f72a815d010 2024-04-26T19:50:30.314 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:50:30.316 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:50:30.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:30.449+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f72a8074330 con 0x7f72a815d010 2024-04-26T19:50:31.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.103+0000 7f7296ffd640 1 -- 172.21.15.79:0/158605497 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f7298058aa0 con 0x7f72a815d010 2024-04-26T19:50:31.104 INFO:teuthology.orchestra.run.smithi079.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T19:50:31.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.107+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f728806c8f0 msgr2=0x7f728806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:31.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.107+0000 7f72accda640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f728806c8f0 0x7f728806edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72a0034000 tx=0x7f72a002e000 comp rx=0 tx=0).stop 2024-04-26T19:50:31.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.107+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 msgr2=0x7f72a814ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:31.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.107+0000 7f72accda640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 0x7f72a814ec50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f729800c4b0 tx=0x7f729800c970 comp rx=0 tx=0).stop 2024-04-26T19:50:31.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.108+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 shutdown_connections 2024-04-26T19:50:31.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.108+0000 7f72accda640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f728806c8f0 0x7f728806edb0 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72a0034000 tx=0x7f72a002e000 comp rx=0 tx=0).stop 2024-04-26T19:50:31.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.108+0000 7f72accda640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f72a815d010 0x7f72a814ec50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:31.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.108+0000 7f72accda640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f72a80d1730 0x7f72a81f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:31.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.108+0000 7f72accda640 1 --2- 172.21.15.79:0/158605497 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f72a806f0e0 0x7f72a8159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:31.109 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.108+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 >> 172.21.15.79:0/158605497 conn(0x7f72a8068ad0 msgr2=0x7f72a815d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:31.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.109+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 shutdown_connections 2024-04-26T19:50:31.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.110+0000 7f72accda640 1 -- 172.21.15.79:0/158605497 wait complete. 2024-04-26T19:50:31.130 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-26T19:50:31.130 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-26T19:50:31.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.234+0000 7f122a924640 1 Processor -- start 2024-04-26T19:50:31.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.234+0000 7f122a924640 1 -- start start 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.235+0000 7f122a924640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f122406dac0 0x7f122406df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.235+0000 7f122a924640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f12240b4b40 0x7f12240b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.235+0000 7f122a924640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12240d2b20 con 0x7f12240b7f50 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.235+0000 7f122a924640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f12240b54f0 con 0x7f122406dac0 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.235+0000 7f122a924640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f12240b8330 con 0x7f12240b4b40 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.235+0000 7f12237fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f122406dac0 0x7f122406df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.235+0000 7f1223fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f12240b7f50 0x7f12240d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57498/0 (socket says 172.21.15.79:57498) 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.235+0000 7f1223fff640 1 -- 172.21.15.79:0/4286221950 learned_addr learned my addr 172.21.15.79:0/4286221950 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.235+0000 7f1228ec1640 1 --2- 172.21.15.79:0/4286221950 >> v2:172.21.15.83:3300/0 conn(0x7f12240b4b40 0x7f12240b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.236+0000 7f12237fe640 1 -- 172.21.15.79:0/4286221950 >> v2:172.21.15.83:3300/0 conn(0x7f12240b4b40 msgr2=0x7f12240b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.236+0000 7f12237fe640 1 --2- 172.21.15.79:0/4286221950 >> v2:172.21.15.83:3300/0 conn(0x7f12240b4b40 0x7f12240b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:31.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.236+0000 7f12237fe640 1 -- 172.21.15.79:0/4286221950 >> v1:172.21.15.79:6789/0 conn(0x7f12240b7f50 legacy=0x7f12240d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:31.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.236+0000 7f12237fe640 1 -- 172.21.15.79:0/4286221950 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12240d2e00 con 0x7f122406dac0 2024-04-26T19:50:31.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.236+0000 7f1222ffd640 1 -- 172.21.15.79:0/4286221950 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1616051035 0 0) 0x7f12240d2b20 con 0x7f12240b7f50 2024-04-26T19:50:31.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.236+0000 7f12237fe640 1 --2- 172.21.15.79:0/4286221950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f122406dac0 0x7f122406df20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f12240b3770 tx=0x7f121802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267f13c6a7daf5aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:31.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.236+0000 7f1222ffd640 1 -- 172.21.15.79:0/4286221950 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218037070 con 0x7f122406dac0 2024-04-26T19:50:31.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.236+0000 7f1222ffd640 1 -- 172.21.15.79:0/4286221950 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f121802add0 con 0x7f122406dac0 2024-04-26T19:50:31.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.236+0000 7f1222ffd640 1 -- 172.21.15.79:0/4286221950 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218032660 con 0x7f122406dac0 2024-04-26T19:50:31.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.237+0000 7f122a924640 1 -- 172.21.15.79:0/4286221950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f122406dac0 msgr2=0x7f122406df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:31.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.237+0000 7f122a924640 1 --2- 172.21.15.79:0/4286221950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f122406dac0 0x7f122406df20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f12240b3770 tx=0x7f121802a080 comp rx=0 tx=0).stop 2024-04-26T19:50:31.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.237+0000 7f122a924640 1 -- 172.21.15.79:0/4286221950 shutdown_connections 2024-04-26T19:50:31.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.237+0000 7f122a924640 1 --2- 172.21.15.79:0/4286221950 >> v2:172.21.15.83:3300/0 conn(0x7f12240b4b40 0x7f12240b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:31.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.237+0000 7f122a924640 1 --2- 172.21.15.79:0/4286221950 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f122406dac0 0x7f122406df20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:31.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.237+0000 7f122a924640 1 -- 172.21.15.79:0/4286221950 >> 172.21.15.79:0/4286221950 conn(0x7f1224071a60 msgr2=0x7f1224071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:31.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.237+0000 7f122a924640 1 -- 172.21.15.79:0/4286221950 shutdown_connections 2024-04-26T19:50:31.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.237+0000 7f122a924640 1 -- 172.21.15.79:0/4286221950 wait complete. 2024-04-26T19:50:31.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.238+0000 7f122a924640 1 Processor -- start 2024-04-26T19:50:31.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.238+0000 7f122a924640 1 -- start start 2024-04-26T19:50:31.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f122a924640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f122406dac0 0x7f122415d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:31.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f122a924640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12240b4b40 0x7f12240cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:31.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f122a924640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12240b7f50 0x7f12240cde40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:31.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f122a924640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f12240cfd40 con 0x7f12240b4b40 2024-04-26T19:50:31.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f122a924640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12240cfaf0 con 0x7f12240b7f50 2024-04-26T19:50:31.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f122a924640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f12240b3ea0 con 0x7f122406dac0 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f12237fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12240b4b40 0x7f12240cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f12237fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12240b4b40 0x7f12240cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53010/0 (socket says 172.21.15.79:53010) 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f12237fe640 1 -- 172.21.15.79:0/563408733 learned_addr learned my addr 172.21.15.79:0/563408733 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f1223fff640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f122406dac0 0x7f122415d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f1228ec1640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12240b7f50 0x7f12240cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f12237fe640 1 -- 172.21.15.79:0/563408733 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f122406dac0 msgr2=0x7f122415d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f12237fe640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f122406dac0 0x7f122415d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f12237fe640 1 -- 172.21.15.79:0/563408733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12240b7f50 msgr2=0x7f12240cde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f12237fe640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12240b7f50 0x7f12240cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f12237fe640 1 -- 172.21.15.79:0/563408733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12241e49f0 con 0x7f12240b4b40 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.239+0000 7f12237fe640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12240b4b40 0x7f12240cd900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1218000c00 tx=0x7f12180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.240+0000 7f1220ff9640 1 -- 172.21.15.79:0/563408733 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218037040 con 0x7f12240b4b40 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.240+0000 7f1220ff9640 1 -- 172.21.15.79:0/563408733 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1218042070 con 0x7f12240b4b40 2024-04-26T19:50:31.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.240+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12241e4cd0 con 0x7f12240b4b40 2024-04-26T19:50:31.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.240+0000 7f1220ff9640 1 -- 172.21.15.79:0/563408733 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121803d660 con 0x7f12240b4b40 2024-04-26T19:50:31.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.240+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12241f65c0 con 0x7f12240b4b40 2024-04-26T19:50:31.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.241+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12240506c0 con 0x7f12240b4b40 2024-04-26T19:50:31.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.241+0000 7f1220ff9640 1 -- 172.21.15.79:0/563408733 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f121803d800 con 0x7f12240b4b40 2024-04-26T19:50:31.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.241+0000 7f1220ff9640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f121006c8f0 0x7f121006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:31.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.241+0000 7f1223fff640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f121006c8f0 0x7f121006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:31.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.241+0000 7f1220ff9640 1 -- 172.21.15.79:0/563408733 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f12180ac150 con 0x7f12240b4b40 2024-04-26T19:50:31.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.242+0000 7f1223fff640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f121006c8f0 0x7f121006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1214000f60 tx=0x7f1214009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:31.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.246+0000 7f1220ff9640 1 -- 172.21.15.79:0/563408733 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1218030e20 con 0x7f12240b4b40 2024-04-26T19:50:31.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:31.430+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f12240b6d10 con 0x7f12240b4b40 2024-04-26T19:50:32.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.091+0000 7f1220ff9640 1 -- 172.21.15.79:0/563408733 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f121807a610 con 0x7f12240b4b40 2024-04-26T19:50:32.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f121006c8f0 msgr2=0x7f121006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:32.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f121006c8f0 0x7f121006edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1214000f60 tx=0x7f1214009040 comp rx=0 tx=0).stop 2024-04-26T19:50:32.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12240b4b40 msgr2=0x7f12240cd900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:32.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12240b4b40 0x7f12240cd900 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1218000c00 tx=0x7f12180046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 shutdown_connections 2024-04-26T19:50:32.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f121006c8f0 0x7f121006edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12240b7f50 0x7f12240cde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f12240b4b40 0x7f12240cd900 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 --2- 172.21.15.79:0/563408733 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f122406dac0 0x7f122415d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 >> 172.21.15.79:0/563408733 conn(0x7f1224071a60 msgr2=0x7f12240b6260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:32.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 shutdown_connections 2024-04-26T19:50:32.096 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.094+0000 7f122a924640 1 -- 172.21.15.79:0/563408733 wait complete. 2024-04-26T19:50:32.107 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T19:50:32.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.212+0000 7fb08b02a640 1 Processor -- start 2024-04-26T19:50:32.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.212+0000 7fb08b02a640 1 -- start start 2024-04-26T19:50:32.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.212+0000 7fb08b02a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb0840b4e10 0x7fb084160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:32.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.212+0000 7fb08b02a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 0x7fb0840b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:32.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.212+0000 7fb08b02a640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0840cd960 con 0x7fb0840b6a70 2024-04-26T19:50:32.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb08b02a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb0840b74d0 con 0x7fb0840b4e10 2024-04-26T19:50:32.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb08b02a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb0840b71f0 con 0x7fb0840b51f0 2024-04-26T19:50:32.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb0895c7640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb0840b6a70 0x7fb0840ce6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57510/0 (socket says 172.21.15.79:57510) 2024-04-26T19:50:32.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb0895c7640 1 -- 172.21.15.79:0/1019517468 learned_addr learned my addr 172.21.15.79:0/1019517468 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb088dc6640 1 --2- 172.21.15.79:0/1019517468 >> v2:172.21.15.83:3300/0 conn(0x7fb0840b4e10 0x7fb084160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb083fff640 1 --2- 172.21.15.79:0/1019517468 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 0x7fb0840b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb0837fe640 1 -- 172.21.15.79:0/1019517468 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3046079675 0 0) 0x7fb07c001490 con 0x7fb0840b6a70 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb0837fe640 1 -- 172.21.15.79:0/1019517468 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb074003520 con 0x7fb0840b6a70 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb088dc6640 1 -- 172.21.15.79:0/1019517468 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 msgr2=0x7fb0840b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb088dc6640 1 --2- 172.21.15.79:0/1019517468 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 0x7fb0840b6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb088dc6640 1 -- 172.21.15.79:0/1019517468 >> v1:172.21.15.79:6789/0 conn(0x7fb0840b6a70 legacy=0x7fb0840ce6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb088dc6640 1 -- 172.21.15.79:0/1019517468 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0840cdbe0 con 0x7fb0840b4e10 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.213+0000 7fb083fff640 1 --2- 172.21.15.79:0/1019517468 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 0x7fb0840b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.214+0000 7fb088dc6640 1 --2- 172.21.15.79:0/1019517468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 0x7fb084160ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb0780097a0 tx=0x7fb07802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a9c5b44b41f36f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.214+0000 7fb0837fe640 1 -- 172.21.15.79:0/1019517468 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078037070 con 0x7fb0840b4e10 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.214+0000 7fb0837fe640 1 -- 172.21.15.79:0/1019517468 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb078032440 con 0x7fb0840b4e10 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.214+0000 7fb0837fe640 1 -- 172.21.15.79:0/1019517468 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078032810 con 0x7fb0840b4e10 2024-04-26T19:50:32.223 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.215+0000 7fb08b02a640 1 -- 172.21.15.79:0/1019517468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 msgr2=0x7fb084160ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.215+0000 7fb08b02a640 1 --2- 172.21.15.79:0/1019517468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 0x7fb084160ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb0780097a0 tx=0x7fb07802a180 comp rx=0 tx=0).stop 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.215+0000 7fb08b02a640 1 -- 172.21.15.79:0/1019517468 shutdown_connections 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.215+0000 7fb08b02a640 1 --2- 172.21.15.79:0/1019517468 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 0x7fb0840b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.215+0000 7fb08b02a640 1 --2- 172.21.15.79:0/1019517468 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 0x7fb084160ab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.215+0000 7fb08b02a640 1 -- 172.21.15.79:0/1019517468 >> 172.21.15.79:0/1019517468 conn(0x7fb084071ab0 msgr2=0x7fb084071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.215+0000 7fb08b02a640 1 -- 172.21.15.79:0/1019517468 shutdown_connections 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.215+0000 7fb08b02a640 1 -- 172.21.15.79:0/1019517468 wait complete. 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.216+0000 7fb08b02a640 1 Processor -- start 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.216+0000 7fb08b02a640 1 -- start start 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb08b02a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 0x7fb084148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb08b02a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 0x7fb084155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb08b02a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0840b6a70 0x7fb0840b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb08b02a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840b6ea0 con 0x7fb0840b6a70 2024-04-26T19:50:32.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb08b02a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb0840ce1a0 con 0x7fb0840b4e10 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb08b02a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb0840cedd0 con 0x7fb0840b51f0 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb0895c7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0840b6a70 0x7fb0840b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb088dc6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 0x7fb084148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb0895c7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0840b6a70 0x7fb0840b5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53018/0 (socket says 172.21.15.79:53018) 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb0895c7640 1 -- 172.21.15.79:0/816889546 learned_addr learned my addr 172.21.15.79:0/816889546 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb083fff640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 0x7fb084155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb088dc6640 1 -- 172.21.15.79:0/816889546 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 msgr2=0x7fb084155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb088dc6640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 0x7fb084155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb088dc6640 1 -- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0840b6a70 msgr2=0x7fb0840b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.217+0000 7fb088dc6640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0840b6a70 0x7fb0840b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.218+0000 7fb088dc6640 1 -- 172.21.15.79:0/816889546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb084149300 con 0x7fb0840b4e10 2024-04-26T19:50:32.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.218+0000 7fb0895c7640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0840b6a70 0x7fb0840b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.218+0000 7fb088dc6640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 0x7fb084148dc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb078031000 tx=0x7fb0780046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.218+0000 7fb0817fa640 1 -- 172.21.15.79:0/816889546 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078041030 con 0x7fb0840b4e10 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.218+0000 7fb0817fa640 1 -- 172.21.15.79:0/816889546 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07803d4e0 con 0x7fb0840b4e10 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.218+0000 7fb0817fa640 1 -- 172.21.15.79:0/816889546 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078037040 con 0x7fb0840b4e10 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.218+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb084157650 con 0x7fb0840b4e10 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.218+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb084157c10 con 0x7fb0840b4e10 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.219+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb084050f10 con 0x7fb0840b4e10 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.220+0000 7fb0817fa640 1 -- 172.21.15.79:0/816889546 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb07803d8b0 con 0x7fb0840b4e10 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.220+0000 7fb0817fa640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb06806c8f0 0x7fb06806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.220+0000 7fb083fff640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb06806c8f0 0x7fb06806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.220+0000 7fb0817fa640 1 -- 172.21.15.79:0/816889546 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7fb0780abdc0 con 0x7fb0840b4e10 2024-04-26T19:50:32.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.221+0000 7fb083fff640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb06806c8f0 0x7fb06806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb070009560 tx=0x7fb070009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:32.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.226+0000 7fb0817fa640 1 -- 172.21.15.79:0/816889546 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb07804b020 con 0x7fb0840b4e10 2024-04-26T19:50:32.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:32.404+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb0840cfcf0 con 0x7fb0840b4e10 2024-04-26T19:50:34.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.121+0000 7fb0817fa640 1 -- 172.21.15.79:0/816889546 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7fb084149300 con 0x7fb0840b4e10 2024-04-26T19:50:34.122 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data_ec' created 2024-04-26T19:50:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.126+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb06806c8f0 msgr2=0x7fb06806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.127+0000 7fb08b02a640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb06806c8f0 0x7fb06806edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb070009560 tx=0x7fb070009190 comp rx=0 tx=0).stop 2024-04-26T19:50:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.127+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 msgr2=0x7fb084148dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.127+0000 7fb08b02a640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 0x7fb084148dc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb078031000 tx=0x7fb0780046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.127+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 shutdown_connections 2024-04-26T19:50:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.127+0000 7fb08b02a640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb06806c8f0 0x7fb06806edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.127+0000 7fb08b02a640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0840b6a70 0x7fb0840b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.127+0000 7fb08b02a640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb0840b51f0 0x7fb084155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.127+0000 7fb08b02a640 1 --2- 172.21.15.79:0/816889546 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb0840b4e10 0x7fb084148dc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.127+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 >> 172.21.15.79:0/816889546 conn(0x7fb084071ab0 msgr2=0x7fb0840d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.128+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 shutdown_connections 2024-04-26T19:50:34.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.128+0000 7fb08b02a640 1 -- 172.21.15.79:0/816889546 wait complete. 2024-04-26T19:50:34.144 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-26T19:50:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.249+0000 7f97aa8bd640 1 Processor -- start 2024-04-26T19:50:34.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.249+0000 7f97aa8bd640 1 -- start start 2024-04-26T19:50:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97aa8bd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a415c080 0x7f97a40cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97aa8bd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f97a406ff50 0x7f97a406df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97aa8bd640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97a40cf280 con 0x7f97a415c460 2024-04-26T19:50:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97aa8bd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f97a40cd7d0 con 0x7f97a406ff50 2024-04-26T19:50:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97aa8bd640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f97a40cf540 con 0x7f97a415c080 2024-04-26T19:50:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a37fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f97a415c460 0x7f97a406f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57512/0 (socket says 172.21.15.79:57512) 2024-04-26T19:50:34.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a37fe640 1 -- 172.21.15.79:0/2333816584 learned_addr learned my addr 172.21.15.79:0/2333816584 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 235615683 0 0) 0x7f97a40cf280 con 0x7f97a415c460 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a3fff640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a415c080 0x7f97a40cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f978c003520 con 0x7f97a415c460 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a8e5a640 1 --2- 172.21.15.79:0/2333816584 >> v2:172.21.15.83:3300/0 conn(0x7f97a406ff50 0x7f97a406df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2545399313 0 0) 0x7f978c003520 con 0x7f97a415c460 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a415c080 msgr2=0x7f97a40cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a415c080 0x7f97a40cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 >> v2:172.21.15.83:3300/0 conn(0x7f97a406ff50 msgr2=0x7f97a406df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 --2- 172.21.15.79:0/2333816584 >> v2:172.21.15.83:3300/0 conn(0x7f97a406ff50 0x7f97a406df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a40cf820 con 0x7f97a415c460 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f9798001670 con 0x7f97a415c460 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 >> v1:172.21.15.79:6789/0 conn(0x7f97a415c460 legacy=0x7f97a406f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f978c0039b0 0x7f978c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a415c080 0x7f978c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:34.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c004d40 0x7f978c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:34.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f97a40cd7d0 con 0x7f978c0039b0 2024-04-26T19:50:34.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97a40cf540 con 0x7f978c004d40 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f978c00b9c0 con 0x7f97a415c080 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.250+0000 7f97a3fff640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f978c0039b0 0x7f978c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.251+0000 7f97a37fe640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a415c080 0x7f978c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.251+0000 7f97a8e5a640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c004d40 0x7f978c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.251+0000 7f97a3fff640 1 -- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a415c080 msgr2=0x7f978c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.251+0000 7f97a3fff640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a415c080 0x7f978c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.251+0000 7f97a3fff640 1 -- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c004d40 msgr2=0x7f978c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.251+0000 7f97a3fff640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c004d40 0x7f978c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.251+0000 7f97a3fff640 1 -- 172.21.15.79:0/2333816584 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c003520 con 0x7f978c0039b0 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.251+0000 7f97a3fff640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f978c0039b0 0x7f978c003dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9794009ac0 tx=0x7f978c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c61667eab9b0ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794036070 con 0x7f978c0039b0 2024-04-26T19:50:34.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9794031400 con 0x7f978c0039b0 2024-04-26T19:50:34.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97a2ffd640 1 -- 172.21.15.79:0/2333816584 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9794009690 con 0x7f978c0039b0 2024-04-26T19:50:34.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97aa8bd640 1 -- 172.21.15.79:0/2333816584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f978c0039b0 msgr2=0x7f978c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97aa8bd640 1 --2- 172.21.15.79:0/2333816584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f978c0039b0 0x7f978c003dd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9794009ac0 tx=0x7f978c005830 comp rx=0 tx=0).stop 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97a8e5a640 1 -- 172.21.15.79:0/2333816584 reap_dead start 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97aa8bd640 1 -- 172.21.15.79:0/2333816584 shutdown_connections 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97aa8bd640 1 -- 172.21.15.79:0/2333816584 >> 172.21.15.79:0/2333816584 conn(0x7f97a4068840 msgr2=0x7f97a40d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97aa8bd640 1 -- 172.21.15.79:0/2333816584 shutdown_connections 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.252+0000 7f97aa8bd640 1 -- 172.21.15.79:0/2333816584 wait complete. 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.253+0000 7f97aa8bd640 1 Processor -- start 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97aa8bd640 1 -- start start 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97aa8bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c0039b0 0x7f97a40b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97aa8bd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97a40b04b0 0x7f97a4147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97aa8bd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a40ae8d0 0x7f97a40aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97aa8bd640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f97a40b01b0 con 0x7f97a40b04b0 2024-04-26T19:50:34.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97aa8bd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f97a40b0030 con 0x7f978c0039b0 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97aa8bd640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f97a40b0330 con 0x7f97a40ae8d0 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97a37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97a40b04b0 0x7f97a4147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97a37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97a40b04b0 0x7f97a4147fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53036/0 (socket says 172.21.15.79:53036) 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97a37fe640 1 -- 172.21.15.79:0/1306087909 learned_addr learned my addr 172.21.15.79:0/1306087909 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97a8e5a640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a40ae8d0 0x7f97a40aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.254+0000 7f97a3fff640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c0039b0 0x7f97a40b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a37fe640 1 -- 172.21.15.79:0/1306087909 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a40ae8d0 msgr2=0x7f97a40aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a37fe640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a40ae8d0 0x7f97a40aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a37fe640 1 -- 172.21.15.79:0/1306087909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c0039b0 msgr2=0x7f97a40b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a37fe640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c0039b0 0x7f97a40b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a37fe640 1 -- 172.21.15.79:0/1306087909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a40b1cd0 con 0x7f97a40b04b0 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a3fff640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c0039b0 0x7f97a40b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a37fe640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97a40b04b0 0x7f97a4147fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9798007370 tx=0x7f9798007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:34.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a0ff9640 1 -- 172.21.15.79:0/1306087909 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979800f9c0 con 0x7f97a40b04b0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a0ff9640 1 -- 172.21.15.79:0/1306087909 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9798018070 con 0x7f97a40b04b0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97a0ff9640 1 -- 172.21.15.79:0/1306087909 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f979800fda0 con 0x7f97a40b04b0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a40b0a20 con 0x7f97a40b04b0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.255+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a40b0fe0 con 0x7f97a40b04b0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.256+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a40506c0 con 0x7f97a40b04b0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.256+0000 7f97a0ff9640 1 -- 172.21.15.79:0/1306087909 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f979801d460 con 0x7f97a40b04b0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.257+0000 7f97a0ff9640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f979006c890 0x7f979006ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.257+0000 7f97a0ff9640 1 -- 172.21.15.79:0/1306087909 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f979808bf60 con 0x7f97a40b04b0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.257+0000 7f97a3fff640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f979006c890 0x7f979006ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.257+0000 7f97a3fff640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f979006c890 0x7f979006ed50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9794033000 tx=0x7f9794008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:34.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.261+0000 7f97a0ff9640 1 -- 172.21.15.79:0/1306087909 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f979805a100 con 0x7f97a40b04b0 2024-04-26T19:50:34.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:34.437+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f97a40d0350 con 0x7f97a40b04b0 2024-04-26T19:50:35.073 INFO:tasks.ceph.mds.d.smithi083.stderr:2024-04-26T19:50:35.072+0000 7fc38c99e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T19:50:35.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.123+0000 7f97a0ff9640 1 -- 172.21.15.79:0/1306087909 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f9798059a00 con 0x7f97a40b04b0 2024-04-26T19:50:35.123 INFO:teuthology.orchestra.run.smithi079.stderr:set pool 4 allow_ec_overwrites to true 2024-04-26T19:50:35.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.127+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f979006c890 msgr2=0x7f979006ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:35.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.127+0000 7f97aa8bd640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f979006c890 0x7f979006ed50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9794033000 tx=0x7f9794008030 comp rx=0 tx=0).stop 2024-04-26T19:50:35.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.127+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97a40b04b0 msgr2=0x7f97a4147fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:35.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.127+0000 7f97aa8bd640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97a40b04b0 0x7f97a4147fd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9798007370 tx=0x7f9798007830 comp rx=0 tx=0).stop 2024-04-26T19:50:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.128+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 shutdown_connections 2024-04-26T19:50:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.128+0000 7f97aa8bd640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f979006c890 0x7f979006ed50 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9794033000 tx=0x7f9794008030 comp rx=0 tx=0).stop 2024-04-26T19:50:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.128+0000 7f97aa8bd640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f97a40ae8d0 0x7f97a40aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.128+0000 7f97aa8bd640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f97a40b04b0 0x7f97a4147fd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.128+0000 7f97aa8bd640 1 --2- 172.21.15.79:0/1306087909 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f978c0039b0 0x7f97a40b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.128+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 >> 172.21.15.79:0/1306087909 conn(0x7f97a4068840 msgr2=0x7f97a40cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:35.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.129+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 shutdown_connections 2024-04-26T19:50:35.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.130+0000 7f97aa8bd640 1 -- 172.21.15.79:0/1306087909 wait complete. 2024-04-26T19:50:35.155 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-26T19:50:35.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.256+0000 7f92c061d640 1 Processor -- start 2024-04-26T19:50:35.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.256+0000 7f92c061d640 1 -- start start 2024-04-26T19:50:35.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92c061d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 0x7f92b80b3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92c061d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92b815e1c0 0x7f92b80d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92c061d640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92b815e7e0 con 0x7f92b80b3d90 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92c061d640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f92b80abb10 con 0x7f92b815e1c0 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92c061d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f92b8161cd0 con 0x7f92b80ab730 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92bebba640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f92b815e1c0 0x7f92b80d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92bdbb8640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f92b80b3d90 0x7f92b815dab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57528/0 (socket says 172.21.15.79:57528) 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92bdbb8640 1 -- 172.21.15.79:0/2610878147 learned_addr learned my addr 172.21.15.79:0/2610878147 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92bebba640 1 -- 172.21.15.79:0/2610878147 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 msgr2=0x7f92b80b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92be3b9640 1 --2- 172.21.15.79:0/2610878147 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 0x7f92b80b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92bd3b7640 1 -- 172.21.15.79:0/2610878147 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 371425352 0 0) 0x7f92b815e7e0 con 0x7f92b80b3d90 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92bebba640 1 --2- 172.21.15.79:0/2610878147 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 0x7f92b80b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:35.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92bebba640 1 -- 172.21.15.79:0/2610878147 >> v1:172.21.15.79:6789/0 conn(0x7f92b80b3d90 legacy=0x7f92b815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.257+0000 7f92bebba640 1 -- 172.21.15.79:0/2610878147 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b8160f10 con 0x7f92b815e1c0 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.258+0000 7f92bebba640 1 --2- 172.21.15.79:0/2610878147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 0x7f92b80d1d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009d90 tx=0x7f92b002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c568f13443c6b003 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.258+0000 7f92bd3b7640 1 -- 172.21.15.79:0/2610878147 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0007740 con 0x7f92b815e1c0 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.258+0000 7f92bd3b7640 1 -- 172.21.15.79:0/2610878147 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b00078e0 con 0x7f92b815e1c0 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.258+0000 7f92bd3b7640 1 -- 172.21.15.79:0/2610878147 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0007cb0 con 0x7f92b815e1c0 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.258+0000 7f92c061d640 1 -- 172.21.15.79:0/2610878147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 msgr2=0x7f92b80d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.258+0000 7f92c061d640 1 --2- 172.21.15.79:0/2610878147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 0x7f92b80d1d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009d90 tx=0x7f92b002a660 comp rx=0 tx=0).stop 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.259+0000 7f92c061d640 1 -- 172.21.15.79:0/2610878147 shutdown_connections 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.259+0000 7f92c061d640 1 --2- 172.21.15.79:0/2610878147 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 0x7f92b80d1d60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.259+0000 7f92c061d640 1 --2- 172.21.15.79:0/2610878147 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 0x7f92b80b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.259+0000 7f92c061d640 1 -- 172.21.15.79:0/2610878147 >> 172.21.15.79:0/2610878147 conn(0x7f92b8073b40 msgr2=0x7f92b8073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.259+0000 7f92c061d640 1 -- 172.21.15.79:0/2610878147 shutdown_connections 2024-04-26T19:50:35.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.259+0000 7f92c061d640 1 -- 172.21.15.79:0/2610878147 wait complete. 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.260+0000 7f92c061d640 1 Processor -- start 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.260+0000 7f92c061d640 1 -- start start 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.260+0000 7f92c061d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 0x7f92b8135b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.260+0000 7f92c061d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92b80b3d90 0x7f92b80ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.260+0000 7f92c061d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 0x7f92b80afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.260+0000 7f92c061d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f92b81617b0 con 0x7f92b815e1c0 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.260+0000 7f92c061d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f92b81614d0 con 0x7f92b80b3d90 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.260+0000 7f92c061d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f92b80d22a0 con 0x7f92b80ab730 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bebba640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 0x7f92b80afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bebba640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 0x7f92b80afc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53064/0 (socket says 172.21.15.79:53064) 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bebba640 1 -- 172.21.15.79:0/1975369511 learned_addr learned my addr 172.21.15.79:0/1975369511 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bdbb8640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92b80b3d90 0x7f92b80ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92be3b9640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 0x7f92b8135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:35.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bebba640 1 -- 172.21.15.79:0/1975369511 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 msgr2=0x7f92b8135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bebba640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 0x7f92b8135b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bebba640 1 -- 172.21.15.79:0/1975369511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92b80b3d90 msgr2=0x7f92b80ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bebba640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92b80b3d90 0x7f92b80ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bebba640 1 -- 172.21.15.79:0/1975369511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b80add70 con 0x7f92b815e1c0 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92bebba640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 0x7f92b80afc80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f92b002ac50 tx=0x7f92b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92aeffd640 1 -- 172.21.15.79:0/1975369511 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b003f070 con 0x7f92b815e1c0 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92aeffd640 1 -- 172.21.15.79:0/1975369511 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b0031c90 con 0x7f92b815e1c0 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b80ae050 con 0x7f92b815e1c0 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92aeffd640 1 -- 172.21.15.79:0/1975369511 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b0044680 con 0x7f92b815e1c0 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.261+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b80ae610 con 0x7f92b815e1c0 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.262+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b80506c0 con 0x7f92b815e1c0 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.263+0000 7f92aeffd640 1 -- 172.21.15.79:0/1975369511 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92b003b410 con 0x7f92b815e1c0 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.263+0000 7f92aeffd640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f929c06c8f0 0x7f929c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:35.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.263+0000 7f92aeffd640 1 -- 172.21.15.79:0/1975369511 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f92b00ad600 con 0x7f92b815e1c0 2024-04-26T19:50:35.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.263+0000 7f92be3b9640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f929c06c8f0 0x7f929c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:35.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.263+0000 7f92be3b9640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f929c06c8f0 0x7f929c06edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92a800aac0 tx=0x7f92a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:35.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.267+0000 7f92aeffd640 1 -- 172.21.15.79:0/1975369511 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f92b007b180 con 0x7f92b815e1c0 2024-04-26T19:50:35.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:35.481+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f92b80d16e0 con 0x7f92b815e1c0 2024-04-26T19:50:36.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.135+0000 7f92aeffd640 1 -- 172.21.15.79:0/1975369511 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7f92b007f480 con 0x7f92b815e1c0 2024-04-26T19:50:36.136 INFO:teuthology.orchestra.run.smithi079.stderr:added data pool 4 to fsmap 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f929c06c8f0 msgr2=0x7f929c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f929c06c8f0 0x7f929c06edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92a800aac0 tx=0x7f92a8008040 comp rx=0 tx=0).stop 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 msgr2=0x7f92b80afc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 0x7f92b80afc80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f92b002ac50 tx=0x7f92b0004a40 comp rx=0 tx=0).stop 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 shutdown_connections 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f929c06c8f0 0x7f929c06edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f92b815e1c0 0x7f92b80afc80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f92b80b3d90 0x7f92b80ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 --2- 172.21.15.79:0/1975369511 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f92b80ab730 0x7f92b8135b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 >> 172.21.15.79:0/1975369511 conn(0x7f92b8073b40 msgr2=0x7f92b80b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 shutdown_connections 2024-04-26T19:50:36.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.137+0000 7f92c061d640 1 -- 172.21.15.79:0/1975369511 wait complete. 2024-04-26T19:50:36.149 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:50:36.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.249+0000 7f77715da640 1 Processor -- start 2024-04-26T19:50:36.250 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.249+0000 7f77715da640 1 -- start start 2024-04-26T19:50:36.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.250+0000 7f77715da640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f776c06fb20 0x7f776c0d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.250+0000 7f77715da640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776c0b1290 con 0x7f776c148fd0 2024-04-26T19:50:36.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.250+0000 7f77715da640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f776c0d4090 con 0x7f776c06ff00 2024-04-26T19:50:36.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.250+0000 7f77715da640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f776c0d4370 con 0x7f776c06fb20 2024-04-26T19:50:36.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.250+0000 7f776b7fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f776c148fd0 0x7f776c0aebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57542/0 (socket says 172.21.15.79:57542) 2024-04-26T19:50:36.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.250+0000 7f776b7fe640 1 -- 172.21.15.79:0/942860556 learned_addr learned my addr 172.21.15.79:0/942860556 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:36.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.250+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2723805396 0 0) 0x7f776c0b1290 con 0x7f776c148fd0 2024-04-26T19:50:36.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.250+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7758003520 con 0x7f776c148fd0 2024-04-26T19:50:36.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.250+0000 7f776affd640 1 --2- 172.21.15.79:0/942860556 >> v2:172.21.15.83:3300/0 conn(0x7f776c06fb20 0x7f776c0d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 629876327 0 0) 0x7f7758003520 con 0x7f776c148fd0 2024-04-26T19:50:36.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 >> v2:172.21.15.83:3300/0 conn(0x7f776c06fb20 msgr2=0x7f776c0d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:36.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 --2- 172.21.15.79:0/942860556 >> v2:172.21.15.83:3300/0 conn(0x7f776c06fb20 0x7f776c0d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 >> v1:172.21.15.83:6789/0 conn(0x7f776c06ff00 legacy=0x7f776c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c0d4650 con 0x7f776c148fd0 2024-04-26T19:50:36.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f77640019f0 con 0x7f776c148fd0 2024-04-26T19:50:36.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 >> v1:172.21.15.79:6789/0 conn(0x7f776c148fd0 legacy=0x7f776c0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77580039e0 0x7f7758003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7758004800 0x7f7758004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77580051c0 0x7f775800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f776c0d4370 con 0x7f77580039e0 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f775800bf60 con 0x7f77580051c0 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f775800c160 con 0x7f7758004800 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776affd640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77580039e0 0x7f7758003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f7763fff640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7758004800 0x7f7758004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776b7fe640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77580051c0 0x7f775800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776affd640 1 -- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7758004800 msgr2=0x7f7758004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776affd640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7758004800 0x7f7758004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776affd640 1 -- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77580051c0 msgr2=0x7f775800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776affd640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77580051c0 0x7f775800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776affd640 1 -- 172.21.15.79:0/942860556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7758003520 con 0x7f77580039e0 2024-04-26T19:50:36.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f7763fff640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7758004800 0x7f7758004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776b7fe640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77580051c0 0x7f775800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.251+0000 7f776b7fe640 1 -- 172.21.15.79:0/942860556 reap_dead start 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.252+0000 7f776affd640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77580039e0 0x7f7758003dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f775c009ac0 tx=0x7f7758005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec7f0b0d5f9ed3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.252+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c036070 con 0x7f77580039e0 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.252+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f775c031400 con 0x7f77580039e0 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.252+0000 7f776a7fc640 1 -- 172.21.15.79:0/942860556 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c009690 con 0x7f77580039e0 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.253+0000 7f77715da640 1 -- 172.21.15.79:0/942860556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77580039e0 msgr2=0x7f7758003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.253+0000 7f77715da640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77580039e0 0x7f7758003dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f775c009ac0 tx=0x7f7758005830 comp rx=0 tx=0).stop 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.253+0000 7f77715da640 1 -- 172.21.15.79:0/942860556 shutdown_connections 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.253+0000 7f77715da640 1 --2- 172.21.15.79:0/942860556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77580039e0 0x7f7758003dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.253+0000 7f77715da640 1 -- 172.21.15.79:0/942860556 >> 172.21.15.79:0/942860556 conn(0x7f776c071c30 msgr2=0x7f776c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.253+0000 7f77715da640 1 -- 172.21.15.79:0/942860556 shutdown_connections 2024-04-26T19:50:36.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.253+0000 7f77715da640 1 -- 172.21.15.79:0/942860556 wait complete. 2024-04-26T19:50:36.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.254+0000 7f77715da640 1 Processor -- start 2024-04-26T19:50:36.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.254+0000 7f77715da640 1 -- start start 2024-04-26T19:50:36.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.254+0000 7f77715da640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77580039e0 0x7f776c146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.254+0000 7f77715da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c144270 0x7f776c15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f77715da640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f776c142380 0x7f776c142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f77715da640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f776c0d2cd0 con 0x7f776c142380 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f77715da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f776c0d4c10 con 0x7f776c144270 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f77715da640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f776c0d2e50 con 0x7f77580039e0 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f776c142380 0x7f776c142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f776c142380 0x7f776c142800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53088/0 (socket says 172.21.15.79:53088) 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776b7fe640 1 -- 172.21.15.79:0/957268425 learned_addr learned my addr 172.21.15.79:0/957268425 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776affd640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77580039e0 0x7f776c146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f7763fff640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c144270 0x7f776c15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776b7fe640 1 -- 172.21.15.79:0/957268425 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77580039e0 msgr2=0x7f776c146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776b7fe640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77580039e0 0x7f776c146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776b7fe640 1 -- 172.21.15.79:0/957268425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c144270 msgr2=0x7f776c15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776b7fe640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c144270 0x7f776c15ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776b7fe640 1 -- 172.21.15.79:0/957268425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776c144aa0 con 0x7f776c142380 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f776b7fe640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f776c142380 0x7f776c142800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f776c0b11e0 tx=0x7f7764007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f77637fe640 1 -- 172.21.15.79:0/957268425 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776400eb00 con 0x7f776c142380 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f77637fe640 1 -- 172.21.15.79:0/957268425 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776400eca0 con 0x7f776c142380 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f77637fe640 1 -- 172.21.15.79:0/957268425 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764014630 con 0x7f776c142380 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776c144d80 con 0x7f776c142380 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.255+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776c140540 con 0x7f776c142380 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.256+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776c140740 con 0x7f776c142380 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.257+0000 7f77637fe640 1 -- 172.21.15.79:0/957268425 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77640147d0 con 0x7f776c142380 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.257+0000 7f77637fe640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f774c06c8f0 0x7f774c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.257+0000 7f776affd640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f774c06c8f0 0x7f774c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.257+0000 7f77637fe640 1 -- 172.21.15.79:0/957268425 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f776408bb80 con 0x7f776c142380 2024-04-26T19:50:36.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.257+0000 7f776affd640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f774c06c8f0 0x7f774c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f775c033000 tx=0x7f775c008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:36.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.261+0000 7f77637fe640 1 -- 172.21.15.79:0/957268425 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f776c140740 con 0x7f776c142380 2024-04-26T19:50:36.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.472+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f776c140330 con 0x7f776c142380 2024-04-26T19:50:36.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.473+0000 7f77637fe640 1 -- 172.21.15.79:0/957268425 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5174 (secure 0 0 0) 0x7f7764059620 con 0x7f776c142380 2024-04-26T19:50:36.474 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:36.474 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:35.482771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:50:36.474 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 7 2024-04-26T19:50:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f774c06c8f0 msgr2=0x7f774c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f774c06c8f0 0x7f774c06edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f775c033000 tx=0x7f775c008030 comp rx=0 tx=0).stop 2024-04-26T19:50:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f776c142380 msgr2=0x7f776c142800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f776c142380 0x7f776c142800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f776c0b11e0 tx=0x7f7764007bc0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 shutdown_connections 2024-04-26T19:50:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f774c06c8f0 0x7f774c06edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f776c142380 0x7f776c142800 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f776c144270 0x7f776c15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 --2- 172.21.15.79:0/957268425 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77580039e0 0x7f776c146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 >> 172.21.15.79:0/957268425 conn(0x7f776c071c30 msgr2=0x7f776c161b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 shutdown_connections 2024-04-26T19:50:36.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.475+0000 7f77715da640 1 -- 172.21.15.79:0/957268425 wait complete. 2024-04-26T19:50:36.487 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:36.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.581+0000 7fd5543e7640 1 Processor -- start 2024-04-26T19:50:36.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.581+0000 7fd5543e7640 1 -- start start 2024-04-26T19:50:36.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.582+0000 7fd5543e7640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c06fb20 0x7fd54c0d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.582+0000 7fd5543e7640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd54c148fd0 0x7fd54c0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.582+0000 7fd5543e7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd54c0b1290 con 0x7fd54c06ff00 2024-04-26T19:50:36.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.582+0000 7fd5543e7640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd54c0d3e60 con 0x7fd54c148fd0 2024-04-26T19:50:36.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.582+0000 7fd5543e7640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c0d4090 con 0x7fd54c06fb20 2024-04-26T19:50:36.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.583+0000 7fd552984640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd54c148fd0 0x7fd54c0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.583+0000 7fd552984640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd54c148fd0 0x7fd54c0d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53090/0 (socket says 172.21.15.79:53090) 2024-04-26T19:50:36.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.583+0000 7fd552984640 1 -- 172.21.15.79:0/3372449144 learned_addr learned my addr 172.21.15.79:0/3372449144 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:36.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.583+0000 7fd552183640 1 --2- 172.21.15.79:0/3372449144 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c06fb20 0x7fd54c0d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.583+0000 7fd552984640 1 -- 172.21.15.79:0/3372449144 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c06fb20 msgr2=0x7fd54c0d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:36.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.583+0000 7fd552984640 1 --2- 172.21.15.79:0/3372449144 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c06fb20 0x7fd54c0d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.583+0000 7fd552984640 1 -- 172.21.15.79:0/3372449144 >> v1:172.21.15.83:6789/0 conn(0x7fd54c06ff00 legacy=0x7fd54c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.583+0000 7fd552984640 1 -- 172.21.15.79:0/3372449144 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c0d4370 con 0x7fd54c148fd0 2024-04-26T19:50:36.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd552984640 1 --2- 172.21.15.79:0/3372449144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c148fd0 0x7fd54c0d3920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009870 tx=0x7fd53c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d7898659aea40c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:36.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd551181640 1 -- 172.21.15.79:0/3372449144 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c037070 con 0x7fd54c148fd0 2024-04-26T19:50:36.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd551181640 1 -- 172.21.15.79:0/3372449144 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd53c02ae50 con 0x7fd54c148fd0 2024-04-26T19:50:36.585 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd551181640 1 -- 172.21.15.79:0/3372449144 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd53c032860 con 0x7fd54c148fd0 2024-04-26T19:50:36.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd5543e7640 1 -- 172.21.15.79:0/3372449144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c148fd0 msgr2=0x7fd54c0d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd5543e7640 1 --2- 172.21.15.79:0/3372449144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c148fd0 0x7fd54c0d3920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd53c009870 tx=0x7fd53c02a300 comp rx=0 tx=0).stop 2024-04-26T19:50:36.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd5543e7640 1 -- 172.21.15.79:0/3372449144 shutdown_connections 2024-04-26T19:50:36.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd5543e7640 1 --2- 172.21.15.79:0/3372449144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c148fd0 0x7fd54c0d3920 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd5543e7640 1 --2- 172.21.15.79:0/3372449144 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c06fb20 0x7fd54c0d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd5543e7640 1 -- 172.21.15.79:0/3372449144 >> 172.21.15.79:0/3372449144 conn(0x7fd54c071c30 msgr2=0x7fd54c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:36.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd5543e7640 1 -- 172.21.15.79:0/3372449144 shutdown_connections 2024-04-26T19:50:36.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.584+0000 7fd5543e7640 1 -- 172.21.15.79:0/3372449144 wait complete. 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.585+0000 7fd5543e7640 1 Processor -- start 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.585+0000 7fd5543e7640 1 -- start start 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd5543e7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c06fb20 0x7fd54c138e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd5543e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd54c06ff00 0x7fd54c144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd5543e7640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c148fd0 0x7fd54c0b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd5543e7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c0d4c10 con 0x7fd54c06fb20 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd5543e7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd54c0d4930 con 0x7fd54c06ff00 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd5543e7640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd54c0aebb0 con 0x7fd54c148fd0 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd552183640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c06fb20 0x7fd54c138e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd552183640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c06fb20 0x7fd54c138e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53102/0 (socket says 172.21.15.79:53102) 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd552183640 1 -- 172.21.15.79:0/4083387020 learned_addr learned my addr 172.21.15.79:0/4083387020 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd552984640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c148fd0 0x7fd54c0b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd551982640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd54c06ff00 0x7fd54c144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd552183640 1 -- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c148fd0 msgr2=0x7fd54c0b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.586+0000 7fd552183640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c148fd0 0x7fd54c0b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd552183640 1 -- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd54c06ff00 msgr2=0x7fd54c144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd552183640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd54c06ff00 0x7fd54c144e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd552183640 1 -- 172.21.15.79:0/4083387020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd54c145780 con 0x7fd54c06fb20 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd552984640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c148fd0 0x7fd54c0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd551982640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd54c06ff00 0x7fd54c144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd552183640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c06fb20 0x7fd54c138e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd544009db0 tx=0x7fd54400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd54affd640 1 -- 172.21.15.79:0/4083387020 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544018020 con 0x7fd54c06fb20 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd54affd640 1 -- 172.21.15.79:0/4083387020 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd544014480 con 0x7fd54c06fb20 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd54c145a60 con 0x7fd54c06fb20 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd54affd640 1 -- 172.21.15.79:0/4083387020 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd544014850 con 0x7fd54c06fb20 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.587+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd54c146020 con 0x7fd54c06fb20 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.588+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd54c0506c0 con 0x7fd54c06fb20 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.588+0000 7fd54affd640 1 -- 172.21.15.79:0/4083387020 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd544005d10 con 0x7fd54c06fb20 2024-04-26T19:50:36.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.589+0000 7fd54affd640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd53806c8f0 0x7fd53806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.589+0000 7fd54affd640 1 -- 172.21.15.79:0/4083387020 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fd54408ca50 con 0x7fd54c06fb20 2024-04-26T19:50:36.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.589+0000 7fd551982640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd53806c8f0 0x7fd53806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.589+0000 7fd551982640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd53806c8f0 0x7fd53806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd54c074680 tx=0x7fd540009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:36.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.593+0000 7fd54affd640 1 -- 172.21.15.79:0/4083387020 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd544022070 con 0x7fd54c06fb20 2024-04-26T19:50:36.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.771+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd54c0737e0 con 0x7fd54c06fb20 2024-04-26T19:50:36.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.772+0000 7fd54affd640 1 -- 172.21.15.79:0/4083387020 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18479 (secure 0 0 0) 0x7fd544010040 con 0x7fd54c06fb20 2024-04-26T19:50:36.773 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:36.773 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":24,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:35.482526+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:36.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd53806c8f0 msgr2=0x7fd53806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd53806c8f0 0x7fd53806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd54c074680 tx=0x7fd540009040 comp rx=0 tx=0).stop 2024-04-26T19:50:36.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c06fb20 msgr2=0x7fd54c138e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c06fb20 0x7fd54c138e00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd544009db0 tx=0x7fd54400c490 comp rx=0 tx=0).stop 2024-04-26T19:50:36.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 shutdown_connections 2024-04-26T19:50:36.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd53806c8f0 0x7fd53806edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd54c148fd0 0x7fd54c0b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd54c06ff00 0x7fd54c144e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 --2- 172.21.15.79:0/4083387020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd54c06fb20 0x7fd54c138e00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.774+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 >> 172.21.15.79:0/4083387020 conn(0x7fd54c071c30 msgr2=0x7fd54c161b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:36.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.775+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 shutdown_connections 2024-04-26T19:50:36.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.775+0000 7fd5543e7640 1 -- 172.21.15.79:0/4083387020 wait complete. 2024-04-26T19:50:36.786 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:36.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.882+0000 7f7158d96640 1 Processor -- start 2024-04-26T19:50:36.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.882+0000 7f7158d96640 1 -- start start 2024-04-26T19:50:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f7158d96640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7154160390 0x7f71540b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f7158d96640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71541637e0 0x7f71540b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f7158d96640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7154144ca0 con 0x7f7154160770 2024-04-26T19:50:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f7158d96640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f7154163e00 con 0x7f7154160390 2024-04-26T19:50:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f7158d96640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7154144f30 con 0x7f71541637e0 2024-04-26T19:50:36.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f715259c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7154160390 0x7f71540b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f715259c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7154160390 0x7f71540b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53116/0 (socket says 172.21.15.79:53116) 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f715259c640 1 -- 172.21.15.79:0/1691700690 learned_addr learned my addr 172.21.15.79:0/1691700690 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f715259c640 1 -- 172.21.15.79:0/1691700690 >> v2:172.21.15.83:3300/0 conn(0x7f71541637e0 msgr2=0x7f71540b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f715259c640 1 --2- 172.21.15.79:0/1691700690 >> v2:172.21.15.83:3300/0 conn(0x7f71541637e0 0x7f71540b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f715259c640 1 -- 172.21.15.79:0/1691700690 >> v1:172.21.15.83:6789/0 conn(0x7f7154160770 legacy=0x7f71541630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.883+0000 7f715259c640 1 -- 172.21.15.79:0/1691700690 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7154145210 con 0x7f7154160390 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f715259c640 1 --2- 172.21.15.79:0/1691700690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 0x7f71540b3b10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7154073a70 tx=0x7f714402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86ed39706523905 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f715159a640 1 -- 172.21.15.79:0/1691700690 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144037070 con 0x7f7154160390 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f715159a640 1 -- 172.21.15.79:0/1691700690 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7144032440 con 0x7f7154160390 2024-04-26T19:50:36.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f715159a640 1 -- 172.21.15.79:0/1691700690 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144032810 con 0x7f7154160390 2024-04-26T19:50:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f7158d96640 1 -- 172.21.15.79:0/1691700690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 msgr2=0x7f71540b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f7158d96640 1 --2- 172.21.15.79:0/1691700690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 0x7f71540b3b10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7154073a70 tx=0x7f714402a1a0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f7158d96640 1 -- 172.21.15.79:0/1691700690 shutdown_connections 2024-04-26T19:50:36.888 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f7158d96640 1 --2- 172.21.15.79:0/1691700690 >> v2:172.21.15.83:3300/0 conn(0x7f71541637e0 0x7f71540b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f7158d96640 1 --2- 172.21.15.79:0/1691700690 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 0x7f71540b3b10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f7158d96640 1 -- 172.21.15.79:0/1691700690 >> 172.21.15.79:0/1691700690 conn(0x7f71540b2720 msgr2=0x7f7154070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f7158d96640 1 -- 172.21.15.79:0/1691700690 shutdown_connections 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.884+0000 7f7158d96640 1 -- 172.21.15.79:0/1691700690 wait complete. 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.886+0000 7f7158d96640 1 Processor -- start 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.886+0000 7f7158d96640 1 -- start start 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.886+0000 7f7158d96640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 0x7f7154141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.886+0000 7f7158d96640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7154160770 0x7f7154161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.886+0000 7f7158d96640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71541637e0 0x7f715415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.886+0000 7f7158d96640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7154145b90 con 0x7f7154160390 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.886+0000 7f7158d96640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7154145a10 con 0x7f71541637e0 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.886+0000 7f7158d96640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f7154169460 con 0x7f7154160770 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.886+0000 7f715259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 0x7f7154141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.889 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f715259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 0x7f7154141e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53120/0 (socket says 172.21.15.79:53120) 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f715259c640 1 -- 172.21.15.79:0/3512795276 learned_addr learned my addr 172.21.15.79:0/3512795276 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f7152d9d640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71541637e0 0x7f715415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f7151d9b640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7154160770 0x7f7154161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f715259c640 1 -- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7154160770 msgr2=0x7f7154161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f715259c640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7154160770 0x7f7154161ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f715259c640 1 -- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71541637e0 msgr2=0x7f715415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f715259c640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71541637e0 0x7f715415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f715259c640 1 -- 172.21.15.79:0/3512795276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715415e130 con 0x7f7154160390 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f7152d9d640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71541637e0 0x7f715415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f7151d9b640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7154160770 0x7f7154161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f715259c640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 0x7f7154141e70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f714402a1e0 tx=0x7f71440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f7142ffd640 1 -- 172.21.15.79:0/3512795276 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144041070 con 0x7f7154160390 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f7142ffd640 1 -- 172.21.15.79:0/3512795276 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f714403c870 con 0x7f7154160390 2024-04-26T19:50:36.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715415e410 con 0x7f7154160390 2024-04-26T19:50:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f7142ffd640 1 -- 172.21.15.79:0/3512795276 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7144037040 con 0x7f7154160390 2024-04-26T19:50:36.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.887+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7154141670 con 0x7f7154160390 2024-04-26T19:50:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.888+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71540506c0 con 0x7f7154160390 2024-04-26T19:50:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.889+0000 7f7142ffd640 1 -- 172.21.15.79:0/3512795276 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7144044070 con 0x7f7154160390 2024-04-26T19:50:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.889+0000 7f7142ffd640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f713406c8f0 0x7f713406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.889+0000 7f7151d9b640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f713406c8f0 0x7f713406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.889+0000 7f7142ffd640 1 -- 172.21.15.79:0/3512795276 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f71440acaa0 con 0x7f7154160390 2024-04-26T19:50:36.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.890+0000 7f7151d9b640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f713406c8f0 0x7f713406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7148006fd0 tx=0x7f7148008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:36.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:36.893+0000 7f7142ffd640 1 -- 172.21.15.79:0/3512795276 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f714407ab90 con 0x7f7154160390 2024-04-26T19:50:37.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.070+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f715406d7b0 con 0x7f7154160390 2024-04-26T19:50:37.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.071+0000 7f7142ffd640 1 -- 172.21.15.79:0/3512795276 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18479 (secure 0 0 0) 0x7f714407a490 con 0x7f7154160390 2024-04-26T19:50:37.072 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:37.072 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":24,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:35.482526+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:37.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f713406c8f0 msgr2=0x7f713406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f713406c8f0 0x7f713406edb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7148006fd0 tx=0x7f7148008040 comp rx=0 tx=0).stop 2024-04-26T19:50:37.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 msgr2=0x7f7154141e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 0x7f7154141e70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f714402a1e0 tx=0x7f71440046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 shutdown_connections 2024-04-26T19:50:37.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f713406c8f0 0x7f713406edb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71541637e0 0x7f715415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7154160770 0x7f7154161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 --2- 172.21.15.79:0/3512795276 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7154160390 0x7f7154141e70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 >> 172.21.15.79:0/3512795276 conn(0x7f71540b2720 msgr2=0x7f7154142b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:37.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 shutdown_connections 2024-04-26T19:50:37.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.073+0000 7f7158d96640 1 -- 172.21.15.79:0/3512795276 wait complete. 2024-04-26T19:50:37.101 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:50:37.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.200+0000 7f2b1659e640 1 Processor -- start 2024-04-26T19:50:37.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.200+0000 7f2b1659e640 1 -- start start 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.201+0000 7f2b1659e640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b100b98d0 0x7f2b10071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.201+0000 7f2b1659e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2b1006f4d0 0x7f2b1006f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.201+0000 7f2b1659e640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b1015fc70 con 0x7f2b100b9cb0 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.201+0000 7f2b1659e640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f2b100b4570 con 0x7f2b100b98d0 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.201+0000 7f2b1659e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2b1015ff50 con 0x7f2b1006f4d0 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.201+0000 7f2b1559c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2b100b98d0 0x7f2b10071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.201+0000 7f2b14d9b640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f2b100b9cb0 0x7f2b10073740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57556/0 (socket says 172.21.15.79:57556) 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.201+0000 7f2b14d9b640 1 -- 172.21.15.79:0/1077219827 learned_addr learned my addr 172.21.15.79:0/1077219827 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.201+0000 7f2b15d9d640 1 --2- 172.21.15.79:0/1077219827 >> v2:172.21.15.83:3300/0 conn(0x7f2b1006f4d0 0x7f2b1006f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b1559c640 1 -- 172.21.15.79:0/1077219827 >> v2:172.21.15.83:3300/0 conn(0x7f2b1006f4d0 msgr2=0x7f2b1006f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:37.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b1559c640 1 --2- 172.21.15.79:0/1077219827 >> v2:172.21.15.83:3300/0 conn(0x7f2b1006f4d0 0x7f2b1006f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b1559c640 1 -- 172.21.15.79:0/1077219827 >> v1:172.21.15.79:6789/0 conn(0x7f2b100b9cb0 legacy=0x7f2b10073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b07fff640 1 -- 172.21.15.79:0/1077219827 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3953248300 0 0) 0x7f2b1015fc70 con 0x7f2b100b9cb0 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b1559c640 1 -- 172.21.15.79:0/1077219827 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b10160230 con 0x7f2b100b98d0 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b15d9d640 1 --2- 172.21.15.79:0/1077219827 >> v2:172.21.15.83:3300/0 conn(0x7f2b1006f4d0 0x7f2b1006f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b1559c640 1 --2- 172.21.15.79:0/1077219827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b98d0 0x7f2b10071810 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009d90 tx=0x7f2b0c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5684aa8ee5a1d9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b07fff640 1 -- 172.21.15.79:0/1077219827 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c031600 con 0x7f2b100b98d0 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b07fff640 1 -- 172.21.15.79:0/1077219827 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0c0317a0 con 0x7f2b100b98d0 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.202+0000 7f2b07fff640 1 -- 172.21.15.79:0/1077219827 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c031b70 con 0x7f2b100b98d0 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.203+0000 7f2b1659e640 1 -- 172.21.15.79:0/1077219827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b98d0 msgr2=0x7f2b10071810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.203+0000 7f2b1659e640 1 --2- 172.21.15.79:0/1077219827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b98d0 0x7f2b10071810 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009d90 tx=0x7f2b0c02a610 comp rx=0 tx=0).stop 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.203+0000 7f2b1659e640 1 -- 172.21.15.79:0/1077219827 shutdown_connections 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.203+0000 7f2b1659e640 1 --2- 172.21.15.79:0/1077219827 >> v2:172.21.15.83:3300/0 conn(0x7f2b1006f4d0 0x7f2b1006f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.203+0000 7f2b1659e640 1 --2- 172.21.15.79:0/1077219827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b98d0 0x7f2b10071810 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.203+0000 7f2b1659e640 1 -- 172.21.15.79:0/1077219827 >> 172.21.15.79:0/1077219827 conn(0x7f2b100b4730 msgr2=0x7f2b100b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:37.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.203+0000 7f2b1659e640 1 -- 172.21.15.79:0/1077219827 shutdown_connections 2024-04-26T19:50:37.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.203+0000 7f2b1659e640 1 -- 172.21.15.79:0/1077219827 wait complete. 2024-04-26T19:50:37.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.204+0000 7f2b1659e640 1 Processor -- start 2024-04-26T19:50:37.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.204+0000 7f2b1659e640 1 -- start start 2024-04-26T19:50:37.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b1659e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2b1006f4d0 0x7f2b101f5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b1659e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b100b98d0 0x7f2b100b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b1659e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b9cb0 0x7f2b10153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b1659e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1006fea0 con 0x7f2b100b9cb0 2024-04-26T19:50:37.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b1659e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2b101607f0 con 0x7f2b100b98d0 2024-04-26T19:50:37.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b1659e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1015f8f0 con 0x7f2b1006f4d0 2024-04-26T19:50:37.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b15d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b9cb0 0x7f2b10153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b15d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b9cb0 0x7f2b10153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53128/0 (socket says 172.21.15.79:53128) 2024-04-26T19:50:37.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b15d9d640 1 -- 172.21.15.79:0/4174713176 learned_addr learned my addr 172.21.15.79:0/4174713176 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b14d9b640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b100b98d0 0x7f2b100b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b1559c640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2b1006f4d0 0x7f2b101f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b15d9d640 1 -- 172.21.15.79:0/4174713176 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2b1006f4d0 msgr2=0x7f2b101f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b15d9d640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2b1006f4d0 0x7f2b101f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b15d9d640 1 -- 172.21.15.79:0/4174713176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b100b98d0 msgr2=0x7f2b100b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b15d9d640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b100b98d0 0x7f2b100b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b15d9d640 1 -- 172.21.15.79:0/4174713176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b100af920 con 0x7f2b100b9cb0 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.205+0000 7f2b15d9d640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b9cb0 0x7f2b10153d20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e810 tx=0x7f2b0800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.206+0000 7f2b05ffb640 1 -- 172.21.15.79:0/4174713176 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800f040 con 0x7f2b100b9cb0 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.206+0000 7f2b05ffb640 1 -- 172.21.15.79:0/4174713176 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b08004030 con 0x7f2b100b9cb0 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.206+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b100afc00 con 0x7f2b100b9cb0 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.206+0000 7f2b05ffb640 1 -- 172.21.15.79:0/4174713176 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b080043e0 con 0x7f2b100b9cb0 2024-04-26T19:50:37.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.206+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b100b01c0 con 0x7f2b100b9cb0 2024-04-26T19:50:37.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.207+0000 7f2b05ffb640 1 -- 172.21.15.79:0/4174713176 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2b0800c6d0 con 0x7f2b100b9cb0 2024-04-26T19:50:37.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.207+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b10050f10 con 0x7f2b100b9cb0 2024-04-26T19:50:37.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.207+0000 7f2b05ffb640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2af406c8f0 0x7f2af406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.207+0000 7f2b1559c640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2af406c8f0 0x7f2af406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.207+0000 7f2b05ffb640 1 -- 172.21.15.79:0/4174713176 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2b0808bbc0 con 0x7f2b100b9cb0 2024-04-26T19:50:37.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.208+0000 7f2b1559c640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2af406c8f0 0x7f2af406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c034000 tx=0x7f2b0c02f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:37.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.212+0000 7f2b05ffb640 1 -- 172.21.15.79:0/4174713176 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2b08017020 con 0x7f2b100b9cb0 2024-04-26T19:50:37.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.426+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b10162650 con 0x7f2b100b9cb0 2024-04-26T19:50:37.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.427+0000 7f2b05ffb640 1 -- 172.21.15.79:0/4174713176 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5174 (secure 0 0 0) 0x7f2b0808b070 con 0x7f2b100b9cb0 2024-04-26T19:50:37.428 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:37.428 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:35.482771+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:50:37.428 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 7 2024-04-26T19:50:37.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2af406c8f0 msgr2=0x7f2af406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2af406c8f0 0x7f2af406edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c034000 tx=0x7f2b0c02f000 comp rx=0 tx=0).stop 2024-04-26T19:50:37.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b9cb0 msgr2=0x7f2b10153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b9cb0 0x7f2b10153d20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e810 tx=0x7f2b0800ecd0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 shutdown_connections 2024-04-26T19:50:37.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2af406c8f0 0x7f2af406edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2b100b9cb0 0x7f2b10153d20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2b100b98d0 0x7f2b100b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 --2- 172.21.15.79:0/4174713176 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2b1006f4d0 0x7f2b101f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 >> 172.21.15.79:0/4174713176 conn(0x7f2b100b4730 msgr2=0x7f2b10160a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:37.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 shutdown_connections 2024-04-26T19:50:37.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.429+0000 7f2b1659e640 1 -- 172.21.15.79:0/4174713176 wait complete. 2024-04-26T19:50:37.440 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:37.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.536+0000 7fbdf142a640 1 Processor -- start 2024-04-26T19:50:37.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.536+0000 7fbdf142a640 1 -- start start 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdf142a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbdec06ff00 0x7fbdec148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdf142a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbdec0d2cd0 0x7fbdec0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdf142a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdec0b1290 con 0x7fbdec06fb20 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdf142a640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fbdec0d3140 con 0x7fbdec06ff00 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdf142a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbdec0d3e60 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdea7fc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbdec06ff00 0x7fbdec148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdea7fc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbdec06ff00 0x7fbdec148fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57788/0 (socket says 172.21.15.79:57788) 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdea7fc640 1 -- 172.21.15.79:0/1596315347 learned_addr learned my addr 172.21.15.79:0/1596315347 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdeb7fe640 1 --2- 172.21.15.79:0/1596315347 >> v2:172.21.15.83:3300/0 conn(0x7fbdec0d2cd0 0x7fbdec0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdea7fc640 1 -- 172.21.15.79:0/1596315347 >> v2:172.21.15.83:3300/0 conn(0x7fbdec0d2cd0 msgr2=0x7fbdec0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdea7fc640 1 --2- 172.21.15.79:0/1596315347 >> v2:172.21.15.83:3300/0 conn(0x7fbdec0d2cd0 0x7fbdec0d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdea7fc640 1 -- 172.21.15.79:0/1596315347 >> v1:172.21.15.83:6789/0 conn(0x7fbdec06fb20 legacy=0x7fbdec1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.537+0000 7fbdea7fc640 1 -- 172.21.15.79:0/1596315347 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec0d4090 con 0x7fbdec06ff00 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.538+0000 7fbdea7fc640 1 --2- 172.21.15.79:0/1596315347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec06ff00 0x7fbdec148fd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0b0af0 tx=0x7fbde002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292f8a983baa857b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.538+0000 7fbde9ffb640 1 -- 172.21.15.79:0/1596315347 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0037070 con 0x7fbdec06ff00 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.538+0000 7fbde9ffb640 1 -- 172.21.15.79:0/1596315347 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde0032440 con 0x7fbdec06ff00 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.538+0000 7fbde9ffb640 1 -- 172.21.15.79:0/1596315347 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0032810 con 0x7fbdec06ff00 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.539+0000 7fbdf142a640 1 -- 172.21.15.79:0/1596315347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec06ff00 msgr2=0x7fbdec148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.539+0000 7fbdf142a640 1 --2- 172.21.15.79:0/1596315347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec06ff00 0x7fbdec148fd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0b0af0 tx=0x7fbde002a2c0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.539+0000 7fbdf142a640 1 -- 172.21.15.79:0/1596315347 shutdown_connections 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.539+0000 7fbdf142a640 1 --2- 172.21.15.79:0/1596315347 >> v2:172.21.15.83:3300/0 conn(0x7fbdec0d2cd0 0x7fbdec0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.539+0000 7fbdf142a640 1 --2- 172.21.15.79:0/1596315347 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec06ff00 0x7fbdec148fd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.539+0000 7fbdf142a640 1 -- 172.21.15.79:0/1596315347 >> 172.21.15.79:0/1596315347 conn(0x7fbdec071c30 msgr2=0x7fbdec06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.539+0000 7fbdf142a640 1 -- 172.21.15.79:0/1596315347 shutdown_connections 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.539+0000 7fbdf142a640 1 -- 172.21.15.79:0/1596315347 wait complete. 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.540+0000 7fbdf142a640 1 Processor -- start 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.540+0000 7fbdf142a640 1 -- start start 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdf142a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdec06fb20 0x7fbdec1e7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdf142a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbdec06ff00 0x7fbdec0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdf142a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec0d2cd0 0x7fbdec1e89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdf142a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec0d4930 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdf142a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbdec0d4650 con 0x7fbdec06fb20 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdf142a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbdec0d4c10 con 0x7fbdec06ff00 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec0d2cd0 0x7fbdec1e89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec0d2cd0 0x7fbdec1e89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57790/0 (socket says 172.21.15.79:57790) 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeb7fe640 1 -- 172.21.15.79:0/3256308392 learned_addr learned my addr 172.21.15.79:0/3256308392 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:37.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeaffd640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdec06fb20 0x7fbdec1e7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdea7fc640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbdec06ff00 0x7fbdec0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeb7fe640 1 -- 172.21.15.79:0/3256308392 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbdec06ff00 msgr2=0x7fbdec0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeb7fe640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbdec06ff00 0x7fbdec0b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeb7fe640 1 -- 172.21.15.79:0/3256308392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdec06fb20 msgr2=0x7fbdec1e7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeb7fe640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdec06fb20 0x7fbdec1e7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeb7fe640 1 -- 172.21.15.79:0/3256308392 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdec1f56e0 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdeb7fe640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec0d2cd0 0x7fbdec1e89c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbde400c550 tx=0x7fbde400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdd37fe640 1 -- 172.21.15.79:0/3256308392 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4015070 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdd37fe640 1 -- 172.21.15.79:0/3256308392 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde4004220 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdd37fe640 1 -- 172.21.15.79:0/3256308392 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4003610 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdec1600a0 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.541+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdec160940 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.542+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdec0506c0 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.543+0000 7fbdd37fe640 1 -- 172.21.15.79:0/3256308392 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbde40037b0 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.543+0000 7fbdd37fe640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdd806c8f0 0x7fbdd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:37.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.543+0000 7fbdd37fe640 1 -- 172.21.15.79:0/3256308392 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbde408bb50 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.543+0000 7fbdeaffd640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdd806c8f0 0x7fbdd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:37.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.543+0000 7fbdeaffd640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdd806c8f0 0x7fbdd806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbddc005660 tx=0x7fbddc003d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:37.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.547+0000 7fbdd37fe640 1 -- 172.21.15.79:0/3256308392 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbde40595f0 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.723+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdec072800 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.724+0000 7fbdd37fe640 1 -- 172.21.15.79:0/3256308392 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fbde405d8f0 con 0x7fbdec0d2cd0 2024-04-26T19:50:37.725 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:37.725 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:37.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.726+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdd806c8f0 msgr2=0x7fbdd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.726+0000 7fbdf142a640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdd806c8f0 0x7fbdd806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbddc005660 tx=0x7fbddc003d90 comp rx=0 tx=0).stop 2024-04-26T19:50:37.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.726+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec0d2cd0 msgr2=0x7fbdec1e89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:37.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.726+0000 7fbdf142a640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec0d2cd0 0x7fbdec1e89c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbde400c550 tx=0x7fbde400ca10 comp rx=0 tx=0).stop 2024-04-26T19:50:37.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.726+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 shutdown_connections 2024-04-26T19:50:37.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.726+0000 7fbdf142a640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdd806c8f0 0x7fbdd806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.726+0000 7fbdf142a640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbdec0d2cd0 0x7fbdec1e89c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.726+0000 7fbdf142a640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbdec06ff00 0x7fbdec0b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.726+0000 7fbdf142a640 1 --2- 172.21.15.79:0/3256308392 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbdec06fb20 0x7fbdec1e7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:37.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.727+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 >> 172.21.15.79:0/3256308392 conn(0x7fbdec071c30 msgr2=0x7fbdec072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:37.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.727+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 shutdown_connections 2024-04-26T19:50:37.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:37.727+0000 7fbdf142a640 1 -- 172.21.15.79:0/3256308392 wait complete. 2024-04-26T19:50:38.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T19:50:30.451088+0000', 'modified': '2024-04-26T19:50:35.482771+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4302}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4302': {'gid': 4302, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6835/3159271692', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6833', 'nonce': 3159271692}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3159271692}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4302, 'qdb_cluster': [4302]} 2024-04-26T19:50:38.141 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T19:50:38.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T19:50:38.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.240+0000 7f96b5efc640 1 Processor -- start 2024-04-26T19:50:38.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.240+0000 7f96b5efc640 1 -- start start 2024-04-26T19:50:38.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96b5efc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96b0160390 0x7f96b00b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96b5efc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b01637e0 0x7f96b00b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96b5efc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96b0144ca0 con 0x7f96b0160770 2024-04-26T19:50:38.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96b5efc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f96b0163e00 con 0x7f96b0160390 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96b5efc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f96b0144f30 con 0x7f96b01637e0 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96affff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b01637e0 0x7f96b00b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96af7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f96b0160390 0x7f96b00b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96affff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b01637e0 0x7f96b00b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:54582/0 (socket says 172.21.15.79:54582) 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96affff640 1 -- 172.21.15.79:0/3346247359 learned_addr learned my addr 172.21.15.79:0/3346247359 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.241+0000 7f96aeffd640 1 --1- 172.21.15.79:0/3346247359 >> v1:172.21.15.83:6789/0 conn(0x7f96b0160770 0x7f96b01630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:54540/0 (socket says 172.21.15.79:54540) 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.242+0000 7f96af7fe640 1 -- 172.21.15.79:0/3346247359 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b01637e0 msgr2=0x7f96b00b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.242+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3346247359 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b01637e0 0x7f96b00b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.242+0000 7f96af7fe640 1 -- 172.21.15.79:0/3346247359 >> v1:172.21.15.83:6789/0 conn(0x7f96b0160770 legacy=0x7f96b01630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.242+0000 7f96af7fe640 1 -- 172.21.15.79:0/3346247359 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b0145210 con 0x7f96b0160390 2024-04-26T19:50:38.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.242+0000 7f96affff640 1 --2- 172.21.15.79:0/3346247359 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b01637e0 0x7f96b00b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.242+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3346247359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b0160390 0x7f96b00b3b10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f96a00097a0 tx=0x7f96a002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=555ff3bfc77913ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.242+0000 7f96ae7fc640 1 -- 172.21.15.79:0/3346247359 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0037070 con 0x7f96b0160390 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.242+0000 7f96ae7fc640 1 -- 172.21.15.79:0/3346247359 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a002ae50 con 0x7f96b0160390 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.242+0000 7f96ae7fc640 1 -- 172.21.15.79:0/3346247359 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0032660 con 0x7f96b0160390 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.243+0000 7f96b5efc640 1 -- 172.21.15.79:0/3346247359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b0160390 msgr2=0x7f96b00b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.243+0000 7f96b5efc640 1 --2- 172.21.15.79:0/3346247359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b0160390 0x7f96b00b3b10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f96a00097a0 tx=0x7f96a002a100 comp rx=0 tx=0).stop 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.243+0000 7f96b5efc640 1 -- 172.21.15.79:0/3346247359 shutdown_connections 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.243+0000 7f96b5efc640 1 --2- 172.21.15.79:0/3346247359 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b01637e0 0x7f96b00b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.243+0000 7f96b5efc640 1 --2- 172.21.15.79:0/3346247359 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b0160390 0x7f96b00b3b10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.243+0000 7f96b5efc640 1 -- 172.21.15.79:0/3346247359 >> 172.21.15.79:0/3346247359 conn(0x7f96b00b2720 msgr2=0x7f96b0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.243+0000 7f96b5efc640 1 -- 172.21.15.79:0/3346247359 shutdown_connections 2024-04-26T19:50:38.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.243+0000 7f96b5efc640 1 -- 172.21.15.79:0/3346247359 wait complete. 2024-04-26T19:50:38.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.244+0000 7f96b5efc640 1 Processor -- start 2024-04-26T19:50:38.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.244+0000 7f96b5efc640 1 -- start start 2024-04-26T19:50:38.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96b5efc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b0160390 0x7f96b013f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96b5efc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b0160770 0x7f96b01573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96b5efc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96b01637e0 0x7f96b006f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96b5efc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0145b90 con 0x7f96b01637e0 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96b5efc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0145a10 con 0x7f96b0160390 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96b5efc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f96b0169460 con 0x7f96b0160770 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96affff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96b01637e0 0x7f96b006f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96affff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96b01637e0 0x7f96b006f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57792/0 (socket says 172.21.15.79:57792) 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96affff640 1 -- 172.21.15.79:0/3810087296 learned_addr learned my addr 172.21.15.79:0/3810087296 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96aeffd640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b0160770 0x7f96b01573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b0160390 0x7f96b013f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96affff640 1 -- 172.21.15.79:0/3810087296 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b0160770 msgr2=0x7f96b01573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96affff640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b0160770 0x7f96b01573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.248 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96affff640 1 -- 172.21.15.79:0/3810087296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b0160390 msgr2=0x7f96b013f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96affff640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b0160390 0x7f96b013f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96affff640 1 -- 172.21.15.79:0/3810087296 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b0162220 con 0x7f96b01637e0 2024-04-26T19:50:38.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.245+0000 7f96aeffd640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b0160770 0x7f96b01573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:38.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.246+0000 7f96affff640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96b01637e0 0x7f96b006f280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96a800efc0 tx=0x7f96a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.246+0000 7f9697fff640 1 -- 172.21.15.79:0/3810087296 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a800cbe0 con 0x7f96b01637e0 2024-04-26T19:50:38.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.246+0000 7f9697fff640 1 -- 172.21.15.79:0/3810087296 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a800cd80 con 0x7f96b01637e0 2024-04-26T19:50:38.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.246+0000 7f96b5efc640 1 -- 172.21.15.79:0/3810087296 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b0162500 con 0x7f96b01637e0 2024-04-26T19:50:38.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.246+0000 7f9697fff640 1 -- 172.21.15.79:0/3810087296 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8003bb0 con 0x7f96b01637e0 2024-04-26T19:50:38.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.246+0000 7f96b5efc640 1 -- 172.21.15.79:0/3810087296 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b0162ac0 con 0x7f96b01637e0 2024-04-26T19:50:38.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.248+0000 7f9697fff640 1 -- 172.21.15.79:0/3810087296 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96a801b030 con 0x7f96b01637e0 2024-04-26T19:50:38.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.248+0000 7f9697fff640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f969c06c8f0 0x7f969c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.248+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f969c06c8f0 0x7f969c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.248+0000 7f9697fff640 1 -- 172.21.15.79:0/3810087296 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f96a808c250 con 0x7f96b01637e0 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.248+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f969c06c8f0 0x7f969c06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96a0034000 tx=0x7f96a002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.249+0000 7f96b8c15740 1 Processor -- start 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.249+0000 7f96b8c15740 1 -- start start 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383783440 0x55f383783820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 0x55f3837a1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383759120 0x55f38379f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96b8c15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f3836bd740 con 0x55f383759120 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96b8c15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f3836593a0 con 0x55f383783440 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96b8c15740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x55f3836b9700 con 0x55f3837a0ce0 2024-04-26T19:50:38.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96affff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383759120 0x55f38379f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96affff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383759120 0x55f38379f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57806/0 (socket says 172.21.15.79:57806) 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96affff640 1 -- 172.21.15.79:0/2566415575 learned_addr learned my addr 172.21.15.79:0/2566415575 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96af7fe640 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383783440 0x55f383783820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96aeffd640 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 0x55f3837a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96affff640 1 -- 172.21.15.79:0/2566415575 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 msgr2=0x55f3837a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96affff640 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 0x55f3837a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96affff640 1 -- 172.21.15.79:0/2566415575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383783440 msgr2=0x55f383783820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96affff640 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383783440 0x55f383783820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96affff640 1 -- 172.21.15.79:0/2566415575 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f383675750 con 0x55f383759120 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96af7fe640 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383783440 0x55f383783820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96affff640 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383759120 0x55f38379f3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f96a800cfd0 tx=0x7f96a805a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3ed72a904649ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96957fa640 1 -- 172.21.15.79:0/2566415575 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8062830 con 0x55f383759120 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96957fa640 1 -- 172.21.15.79:0/2566415575 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a80092e0 con 0x55f383759120 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.250+0000 7f96957fa640 1 -- 172.21.15.79:0/2566415575 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8057f50 con 0x55f383759120 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.251+0000 7f96b8c15740 1 -- 172.21.15.79:0/2566415575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383759120 msgr2=0x55f38379f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.251+0000 7f96b8c15740 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383759120 0x55f38379f3a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f96a800cfd0 tx=0x7f96a805a490 comp rx=0 tx=0).stop 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.251+0000 7f96b8c15740 1 -- 172.21.15.79:0/2566415575 shutdown_connections 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.251+0000 7f96b8c15740 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383759120 0x55f38379f3a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.251+0000 7f96b8c15740 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 0x55f3837a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.251+0000 7f96b8c15740 1 --2- 172.21.15.79:0/2566415575 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383783440 0x55f383783820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.251+0000 7f96b8c15740 1 -- 172.21.15.79:0/2566415575 >> 172.21.15.79:0/2566415575 conn(0x55f383755a50 msgr2=0x55f38379e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.251+0000 7f96b8c15740 1 -- 172.21.15.79:0/2566415575 shutdown_connections 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.251+0000 7f96b8c15740 1 -- 172.21.15.79:0/2566415575 wait complete. 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96b8c15740 1 -- start start 2024-04-26T19:50:38.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96b8c15740 1 Processor -- start 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f38377f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3837a0ce0 0x55f383785fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383752670 0x55f383752af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96b8c15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f383759ec0 con 0x55f383783440 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96b8c15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f383644fe0 con 0x55f3837a0ce0 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96b8c15740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x55f383753980 con 0x55f383752670 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96b8c15740 10 client.0 _ll_register_callbacks cb 0x55f38365b150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.252+0000 7f96af7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f38377f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96af7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f38377f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57818/0 (socket says 172.21.15.79:57818) 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96af7fe640 1 -- 172.21.15.79:0/1605695773 learned_addr learned my addr 172.21.15.79:0/1605695773 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96aeffd640 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3837a0ce0 0x55f383785fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96affff640 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383752670 0x55f383752af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96af7fe640 1 -- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383752670 msgr2=0x55f383752af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96af7fe640 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383752670 0x55f383752af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96af7fe640 1 -- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3837a0ce0 msgr2=0x55f383785fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96af7fe640 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3837a0ce0 0x55f383785fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96af7fe640 1 -- 172.21.15.79:0/1605695773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3833cf2d0 con 0x55f383783440 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96af7fe640 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f38377f590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f96a002a6f0 tx=0x7f96a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c7334b82d62b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f968e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f968e7fc640 1 -- 172.21.15.79:0/1605695773 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a004d060 con 0x55f383783440 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f968e7fc640 1 -- 172.21.15.79:0/1605695773 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f96a003eaf0 con 0x55f383783440 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96b8c15740 1 -- 172.21.15.79:0/1605695773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a80092e0 con 0x55f383783440 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f968e7fc640 1 -- 172.21.15.79:0/1605695773 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f96a0053050 con 0x55f383783440 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96b8c15740 10 client.4443 fetch_fsmap learned FSMap version 7 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96b8c15740 1 -- 172.21.15.79:0/1605695773 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55f3834996b0 con 0x55f383783440 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f968e7fc640 1 -- 172.21.15.79:0/1605695773 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a80092e0 con 0x55f383783440 2024-04-26T19:50:38.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f968e7fc640 1 -- 172.21.15.79:0/1605695773 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f96a0056020 con 0x55f383783440 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96b8c15740 10 client.4443 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96b8c15740 10 client.4443 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96b8c15740 20 client.4443 populate_metadata read hostname 'smithi079' 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.253+0000 7f96b8c15740 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55f3837b1d90 0x55f3837b2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.254+0000 7f96b8c15740 4 client.4443 mds_command: new command op to 4302 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.254+0000 7f96b8c15740 1 -- 172.21.15.79:0/1605695773 --> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3836cb350 con 0x55f3837b1d90 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.254+0000 7f96aeffd640 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55f3837b1d90 0x55f3837b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.254+0000 7f968e7fc640 10 client.4443 ms_handle_connect on v2:172.21.15.83:6833/3159271692 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.254+0000 7f96aeffd640 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55f3837b1d90 0x55f3837b2170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4d4c02583d5719b server_cookie=aaff8ee9340346f1 in_seq=0 out_seq=0 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f968e7fc640 1 -- 172.21.15.79:0/1605695773 <== mds.0 v2:172.21.15.83:6833/3159271692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3836cb350 con 0x55f3837b1d90 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f968e7fc640 10 client.4443 handle_command_reply: tid=0 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 client.4443 shutdown 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 -- 172.21.15.79:0/1605695773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 msgr2=0x55f38377f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f38377f590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f96a002a6f0 tx=0x7f96a0004730 comp rx=0 tx=0).stop 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 -- 172.21.15.79:0/1605695773 shutdown_connections 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55f3837b1d90 0x55f3837b2170 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383752670 0x55f383752af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f3837a0ce0 0x55f383785fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f968e7fc640 0 client.4443 ms_handle_reset on v2:172.21.15.83:6833/3159271692 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 --2- 172.21.15.79:0/1605695773 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f38377f590 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 -- 172.21.15.79:0/1605695773 >> 172.21.15.79:0/1605695773 conn(0x55f383759120 msgr2=0x55f3837a17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 -- 172.21.15.79:0/1605695773 shutdown_connections 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 1 -- 172.21.15.79:0/1605695773 wait complete. 2024-04-26T19:50:38.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.255+0000 7f96b8c15740 20 client.4443 trim_cache size 0 max 16384 2024-04-26T19:50:38.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.273+0000 7f96b8c15740 1 Processor -- start 2024-04-26T19:50:38.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.273+0000 7f96b8c15740 1 -- start start 2024-04-26T19:50:38.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383783440 0x55f383782b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f3837a0ce0 0x55f38379de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96b8c15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0201f80 con 0x55f3837a0ce0 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96b8c15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0161a70 con 0x55f383752670 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96b8c15740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f96b0201930 con 0x55f383783440 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96affff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f3837a0ce0 0x55f38379de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96affff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f3837a0ce0 0x55f38379de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57828/0 (socket says 172.21.15.79:57828) 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96affff640 1 -- 172.21.15.79:0/3469540469 learned_addr learned my addr 172.21.15.79:0/3469540469 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96aeffd640 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383783440 0x55f383782b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96affff640 1 -- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383783440 msgr2=0x55f383782b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96affff640 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383783440 0x55f383782b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96affff640 1 -- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 msgr2=0x55f3837b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96affff640 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96affff640 1 -- 172.21.15.79:0/3469540469 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f383754f10 con 0x55f3837a0ce0 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96aeffd640 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383783440 0x55f383782b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.274+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96affff640 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f3837a0ce0 0x55f38379de20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96a8059be0 tx=0x7f96a805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b8fd37db0e4a27 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f9694ff9640 1 -- 172.21.15.79:0/3469540469 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8062830 con 0x55f3837a0ce0 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f9694ff9640 1 -- 172.21.15.79:0/3469540469 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a8076030 con 0x55f3837a0ce0 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f9694ff9640 1 -- 172.21.15.79:0/3469540469 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a80092e0 con 0x55f3837a0ce0 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96b8c15740 1 -- 172.21.15.79:0/3469540469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f3837a0ce0 msgr2=0x55f38379de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f3837a0ce0 0x55f38379de20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f96a8059be0 tx=0x7f96a805b000 comp rx=0 tx=0).stop 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96b8c15740 1 -- 172.21.15.79:0/3469540469 shutdown_connections 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f3837a0ce0 0x55f38379de20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f383783440 0x55f383782b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3469540469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96b8c15740 1 -- 172.21.15.79:0/3469540469 >> 172.21.15.79:0/3469540469 conn(0x55f383759120 msgr2=0x55f383796980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96b8c15740 1 -- 172.21.15.79:0/3469540469 shutdown_connections 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.275+0000 7f96b8c15740 1 -- 172.21.15.79:0/3469540469 wait complete. 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.276+0000 7f96b8c15740 1 -- start start 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.276+0000 7f96b8c15740 1 Processor -- start 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f3837b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 0x55f38379eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55f38377ffb0 con 0x55f383783440 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55f383797a10 con 0x55f383752670 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x55f3837957d0 con 0x55f3837a0ce0 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 10 client.0 _ll_register_callbacks cb 0x7f96a0035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96aeffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f3837b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96aeffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f3837b4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57834/0 (socket says 172.21.15.79:57834) 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96aeffd640 1 -- 172.21.15.79:0/3036154829 learned_addr learned my addr 172.21.15.79:0/3036154829 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96affff640 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 0x55f38379eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96aeffd640 1 -- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 msgr2=0x55f38379eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96aeffd640 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 0x55f38379eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96aeffd640 1 -- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 msgr2=0x55f3837b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96aeffd640 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96aeffd640 1 -- 172.21.15.79:0/3036154829 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f383794530 con 0x55f383783440 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96aeffd640 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f3837b4f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96a4000f60 tx=0x7f96a40073e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e9d5232bac3df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f968e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f968e7fc640 1 -- 172.21.15.79:0/3036154829 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a4026060 con 0x55f383783440 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f968e7fc640 1 -- 172.21.15.79:0/3036154829 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f96a4019750 con 0x55f383783440 2024-04-26T19:50:38.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 1 -- 172.21.15.79:0/3036154829 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96a8076030 con 0x55f383783440 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f968e7fc640 1 -- 172.21.15.79:0/3036154829 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f96a4010040 con 0x55f383783440 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 10 client.4449 fetch_fsmap learned FSMap version 7 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f96b8c15740 1 -- 172.21.15.79:0/3036154829 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x55f3837ad110 con 0x55f383783440 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.277+0000 7f968e7fc640 1 -- 172.21.15.79:0/3036154829 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a8076030 con 0x55f383783440 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f968e7fc640 1 -- 172.21.15.79:0/3036154829 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 7) ==== 3502+0+0 (secure 0 0 0) 0x7f96a4020030 con 0x55f383783440 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96b8c15740 10 client.4449 fetch_fsmap finished waiting for FSMap version 7 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96b8c15740 10 client.4449 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96b8c15740 20 client.4449 populate_metadata read hostname 'smithi079' 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55f383821ed0 0x55f3838242c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96b8c15740 4 client.4449 mds_command: new command op to 4302 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96b8c15740 1 -- 172.21.15.79:0/3036154829 --> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] -- command(tid 0: {"prefix": "status"}) -- 0x55f3836c51c0 con 0x55f383821ed0 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55f383821ed0 0x55f3838242c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f968e7fc640 10 client.4449 ms_handle_connect on v2:172.21.15.83:6833/3159271692 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96af7fe640 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55f383821ed0 0x55f3838242c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d021539b37792a5 server_cookie=33476e5de020b912 in_seq=0 out_seq=0 2024-04-26T19:50:38.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f968e7fc640 1 -- 172.21.15.79:0/3036154829 <== mds.0 v2:172.21.15.83:6833/3159271692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55f3836c51c0 con 0x55f383821ed0 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f968e7fc640 10 client.4449 handle_command_reply: tid=0 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96b8c15740 1 client.4449 shutdown 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96b8c15740 1 -- 172.21.15.79:0/3036154829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 msgr2=0x55f3837b4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.278+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f3837b4f40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96a4000f60 tx=0x7f96a40073e0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b8c15740 1 -- 172.21.15.79:0/3036154829 shutdown_connections 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55f383821ed0 0x55f3838242c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55f3837a0ce0 0x55f38379eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55f383783440 0x55f3837b4f40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f968e7fc640 0 client.4449 ms_handle_reset on v2:172.21.15.83:6833/3159271692 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b8c15740 1 --2- 172.21.15.79:0/3036154829 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55f383752670 0x55f3837b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b8c15740 1 -- 172.21.15.79:0/3036154829 >> 172.21.15.79:0/3036154829 conn(0x55f383759120 msgr2=0x55f383796980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b8c15740 1 -- 172.21.15.79:0/3036154829 shutdown_connections 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b8c15740 1 -- 172.21.15.79:0/3036154829 wait complete. 2024-04-26T19:50:38.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b8c15740 20 client.4449 trim_cache size 0 max 16384 2024-04-26T19:50:38.287 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:50:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "b372f41f-f07b-4764-9bef-bcff62f9a5d2", 2024-04-26T19:50:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2024-04-26T19:50:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4302, 2024-04-26T19:50:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-04-26T19:50:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-04-26T19:50:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-04-26T19:50:38.287 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 7.1735630339999998, 2024-04-26T19:50:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 7, 2024-04-26T19:50:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 25, 2024-04-26T19:50:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 24, 2024-04-26T19:50:38.288 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 14.197900990000001 2024-04-26T19:50:38.288 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 -- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f969c06c8f0 msgr2=0x7f969c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f969c06c8f0 0x7f969c06edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96a0034000 tx=0x7f96a002e000 comp rx=0 tx=0).stop 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 -- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96b01637e0 msgr2=0x7f96b006f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96b01637e0 0x7f96b006f280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f96a800efc0 tx=0x7f96a800c490 comp rx=0 tx=0).stop 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 -- 172.21.15.79:0/3810087296 shutdown_connections 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f969c06c8f0 0x7f969c06edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f96b01637e0 0x7f96b006f280 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f96b0160770 0x7f96b01573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 --2- 172.21.15.79:0/3810087296 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f96b0160390 0x7f96b013f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 -- 172.21.15.79:0/3810087296 >> 172.21.15.79:0/3810087296 conn(0x7f96b00b2720 msgr2=0x7f96b0143610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 -- 172.21.15.79:0/3810087296 shutdown_connections 2024-04-26T19:50:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.279+0000 7f96b5efc640 1 -- 172.21.15.79:0/3810087296 wait complete. 2024-04-26T19:50:38.291 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T19:50:38.291 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T19:50:38.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T19:50:38.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.387+0000 7f7edc0d3640 1 Processor -- start 2024-04-26T19:50:38.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.387+0000 7f7edc0d3640 1 -- start start 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7edc0d3640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7ed406fb20 0x7f7ed40d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7edc0d3640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7ed4148fd0 0x7f7ed40d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7edc0d3640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ed40b1290 con 0x7f7ed406ff00 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7edc0d3640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed40d3e60 con 0x7f7ed406fb20 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7edc0d3640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7ed40d4090 con 0x7f7ed4148fd0 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7ed9e6f640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7ed406fb20 0x7f7ed40d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7ed966e640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7ed406ff00 0x7f7ed41488c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44302/0 (socket says 172.21.15.79:44302) 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7ed966e640 1 -- 172.21.15.79:0/396913938 learned_addr learned my addr 172.21.15.79:0/396913938 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7eda670640 1 --2- 172.21.15.79:0/396913938 >> v2:172.21.15.83:3300/0 conn(0x7f7ed4148fd0 0x7f7ed40d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7ed9e6f640 1 -- 172.21.15.79:0/396913938 >> v2:172.21.15.83:3300/0 conn(0x7f7ed4148fd0 msgr2=0x7f7ed40d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:38.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7ed9e6f640 1 --2- 172.21.15.79:0/396913938 >> v2:172.21.15.83:3300/0 conn(0x7f7ed4148fd0 0x7f7ed40d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7ed9e6f640 1 -- 172.21.15.79:0/396913938 >> v1:172.21.15.79:6789/0 conn(0x7f7ed406ff00 legacy=0x7f7ed41488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.388+0000 7f7ed8e6d640 1 -- 172.21.15.79:0/396913938 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265007578 0 0) 0x7f7ed40b1290 con 0x7f7ed406ff00 2024-04-26T19:50:38.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.389+0000 7f7ed9e6f640 1 -- 172.21.15.79:0/396913938 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed40d4370 con 0x7f7ed406fb20 2024-04-26T19:50:38.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.389+0000 7f7eda670640 1 --2- 172.21.15.79:0/396913938 >> v2:172.21.15.83:3300/0 conn(0x7f7ed4148fd0 0x7f7ed40d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:38.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.389+0000 7f7ed9e6f640 1 --2- 172.21.15.79:0/396913938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 0x7f7ed40d2cd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009dc0 tx=0x7f7ed002a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4181641a2644a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.389+0000 7f7ed8e6d640 1 -- 172.21.15.79:0/396913938 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0007740 con 0x7f7ed406fb20 2024-04-26T19:50:38.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.389+0000 7f7ed8e6d640 1 -- 172.21.15.79:0/396913938 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed00078e0 con 0x7f7ed406fb20 2024-04-26T19:50:38.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.389+0000 7f7ed8e6d640 1 -- 172.21.15.79:0/396913938 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0007cb0 con 0x7f7ed406fb20 2024-04-26T19:50:38.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.390+0000 7f7edc0d3640 1 -- 172.21.15.79:0/396913938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 msgr2=0x7f7ed40d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.390+0000 7f7edc0d3640 1 --2- 172.21.15.79:0/396913938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 0x7f7ed40d2cd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009dc0 tx=0x7f7ed002a760 comp rx=0 tx=0).stop 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.390+0000 7f7edc0d3640 1 -- 172.21.15.79:0/396913938 shutdown_connections 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.390+0000 7f7edc0d3640 1 --2- 172.21.15.79:0/396913938 >> v2:172.21.15.83:3300/0 conn(0x7f7ed4148fd0 0x7f7ed40d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.390+0000 7f7edc0d3640 1 --2- 172.21.15.79:0/396913938 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 0x7f7ed40d2cd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.390+0000 7f7edc0d3640 1 -- 172.21.15.79:0/396913938 >> 172.21.15.79:0/396913938 conn(0x7f7ed4071c30 msgr2=0x7f7ed406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.390+0000 7f7edc0d3640 1 -- 172.21.15.79:0/396913938 shutdown_connections 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.390+0000 7f7edc0d3640 1 -- 172.21.15.79:0/396913938 wait complete. 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.391+0000 7f7edc0d3640 1 Processor -- start 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.391+0000 7f7edc0d3640 1 -- start start 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7edc0d3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 0x7f7ed40c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7edc0d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed406ff00 0x7f7ed4146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7edc0d3640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7ed4148fd0 0x7f7ed40b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7edc0d3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed40d4c10 con 0x7f7ed406fb20 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7edc0d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed40d4930 con 0x7f7ed406ff00 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7edc0d3640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed40aebb0 con 0x7f7ed4148fd0 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7ed9e6f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 0x7f7ed40c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7ed9e6f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 0x7f7ed40c5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57852/0 (socket says 172.21.15.79:57852) 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7ed9e6f640 1 -- 172.21.15.79:0/3162293184 learned_addr learned my addr 172.21.15.79:0/3162293184 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7ed9e6f640 1 -- 172.21.15.79:0/3162293184 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7ed4148fd0 msgr2=0x7f7ed40b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7ed9e6f640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7ed4148fd0 0x7f7ed40b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7ed9e6f640 1 -- 172.21.15.79:0/3162293184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed406ff00 msgr2=0x7f7ed4146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7ed9e6f640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed406ff00 0x7f7ed4146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7ed9e6f640 1 -- 172.21.15.79:0/3162293184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed415f090 con 0x7f7ed406fb20 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7ed9e6f640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 0x7f7ed40c5fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009ee0 tx=0x7f7ed0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7eca7fc640 1 -- 172.21.15.79:0/3162293184 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0040070 con 0x7f7ed406fb20 2024-04-26T19:50:38.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7eca7fc640 1 -- 172.21.15.79:0/3162293184 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ed0031bc0 con 0x7f7ed406fb20 2024-04-26T19:50:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed4147430 con 0x7f7ed406fb20 2024-04-26T19:50:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7eca7fc640 1 -- 172.21.15.79:0/3162293184 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0045460 con 0x7f7ed406fb20 2024-04-26T19:50:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.392+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed40c54d0 con 0x7f7ed406fb20 2024-04-26T19:50:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.393+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed40506c0 con 0x7f7ed406fb20 2024-04-26T19:50:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.394+0000 7f7eca7fc640 1 -- 172.21.15.79:0/3162293184 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ed0039040 con 0x7f7ed406fb20 2024-04-26T19:50:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.394+0000 7f7eca7fc640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7eb806c8f0 0x7f7eb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.394+0000 7f7ed966e640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7eb806c8f0 0x7f7eb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.394+0000 7f7eca7fc640 1 -- 172.21.15.79:0/3162293184 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7ed00ad790 con 0x7f7ed406fb20 2024-04-26T19:50:38.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.395+0000 7f7ed966e640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7eb806c8f0 0x7f7eb806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4007fd0 tx=0x7f7ec4007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:38.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.399+0000 7f7eca7fc640 1 -- 172.21.15.79:0/3162293184 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7ed007b880 con 0x7f7ed406fb20 2024-04-26T19:50:38.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.613+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7ed406ec70 con 0x7f7ed406fb20 2024-04-26T19:50:38.613 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.613+0000 7f7eca7fc640 1 -- 172.21.15.79:0/3162293184 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7f7ed007b180 con 0x7f7ed406fb20 2024-04-26T19:50:38.614 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T19:50:38.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7eb806c8f0 msgr2=0x7f7eb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7eb806c8f0 0x7f7eb806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4007fd0 tx=0x7f7ec4007920 comp rx=0 tx=0).stop 2024-04-26T19:50:38.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 msgr2=0x7f7ed40c5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:38.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 0x7f7ed40c5fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009ee0 tx=0x7f7ed0004a40 comp rx=0 tx=0).stop 2024-04-26T19:50:38.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 shutdown_connections 2024-04-26T19:50:38.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7eb806c8f0 0x7f7eb806edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7ed4148fd0 0x7f7ed40b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7ed406ff00 0x7f7ed4146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 --2- 172.21.15.79:0/3162293184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7ed406fb20 0x7f7ed40c5fb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:38.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 >> 172.21.15.79:0/3162293184 conn(0x7f7ed4071c30 msgr2=0x7f7ed4161c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:38.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 shutdown_connections 2024-04-26T19:50:38.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38.615+0000 7f7edc0d3640 1 -- 172.21.15.79:0/3162293184 wait complete. 2024-04-26T19:50:38.626 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T19:50:38.626 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T19:50:38.626 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2024-04-26T19:50:38.626 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-26T19:50:38.626 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-26T19:50:38.626 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T19:50:38.626 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T19:50:38.626 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T19:50:38.627 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T19:50:38.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:38.627 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f0 2024-04-26T19:50:38.642 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f1 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42710sec preferred_lft 42710sec 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::5909:6a14:c050:be1a/64 scope link noprefixroute 2024-04-26T19:50:38.643 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:50:38.644 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T19:50:38.644 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:50:38.644 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:50:38.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add name ceph-brx type bridge 2024-04-26T19:50:38.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr flush dev ceph-brx 2024-04-26T19:50:38.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set ceph-brx up 2024-04-26T19:50:38.644 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T19:50:38.644 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:50:38.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:50:38.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:50:38.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:38.894 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T19:50:38.918 INFO:teuthology.orchestra.run.smithi079.stdout:1 2024-04-26T19:50:38.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:38.920 DEBUG:teuthology.orchestra.run.smithi079:> route 2024-04-26T19:50:38.986 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2024-04-26T19:50:38.986 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T19:50:38.986 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T19:50:38.986 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T19:50:38.986 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T19:50:38.987 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:50:38.987 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:50:38.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T19:50:38.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T19:50:38.987 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T19:50:38.987 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:50:39.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:50:39.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:50:39.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:39.276 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-04-26T19:50:39.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:39.292 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2024-04-26T19:50:39.346 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:50:39.347 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:50:39.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T19:50:39.347 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-26T19:50:39.347 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:50:39.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:50:39.456 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:50:39.462 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-26T19:50:39.462 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:50:39.462 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:50:39.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-26T19:50:39.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T19:50:39.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-26T19:50:39.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-26T19:50:39.463 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-26T19:50:39.463 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:50:39.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:50:39.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:50:39.580 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:50:39.580 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:50:39.580 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2024-04-26T19:50:39.581 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T19:50:39.581 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:50:39.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:50:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:50:39.655 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-26T19:50:39.656 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:50:39.656 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-26T19:50:39.671 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-26T19:50:39.671 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:50:39.672 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-26T19:50:39.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2024-04-26T19:50:39.790 DEBUG:teuthology.orchestra.run.smithi079:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T19:50:39.846 INFO:teuthology.orchestra.run.smithi079.stdout:/proc 2024-04-26T19:50:39.846 INFO:teuthology.orchestra.run.smithi079.stdout:/sys 2024-04-26T19:50:39.846 INFO:teuthology.orchestra.run.smithi079.stdout:/dev 2024-04-26T19:50:39.846 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/security 2024-04-26T19:50:39.846 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/shm 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/pts 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/run 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/cgroup 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/pstore 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/bpf 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/ 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/selinux 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/hugepages 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/mqueue 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/debug 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/tracing 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/config 2024-04-26T19:50:39.847 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/fuse/connections 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/run/user/1000 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T19:50:39.848 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T19:50:39.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:39.849 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2024-04-26T19:50:39.901 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-26T19:50:39.901 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-26T19:50:39.944 DEBUG:teuthology.orchestra.run.smithi079:> sudo modprobe fuse 2024-04-26T19:50:39.966 DEBUG:teuthology.orchestra.run.smithi079:> cat /proc/self/mounts | awk '{print $2}' 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/proc 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/sys 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/dev 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/security 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/shm 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/pts 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/run 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/cgroup 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/pstore 2024-04-26T19:50:40.024 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/bpf 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/ 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/selinux 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/proc/sys/fs/binfmt_misc 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/hugepages 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/dev/mqueue 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/debug 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/tracing 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-sysusers.service 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/kernel/config 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/sys/fs/fuse/connections 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-sysctl.service 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/nfs/rpc_pipefs 2024-04-26T19:50:40.025 INFO:teuthology.orchestra.run.smithi079.stdout:/run/user/1000 2024-04-26T19:50:40.026 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph 2024-04-26T19:50:40.026 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph/osd/ceph-0 2024-04-26T19:50:40.026 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph/osd/ceph-1 2024-04-26T19:50:40.026 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph/osd/ceph-2 2024-04-26T19:50:40.026 INFO:teuthology.orchestra.run.smithi079.stdout:/var/lib/ceph/osd/ceph-3 2024-04-26T19:50:40.026 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns 2024-04-26T19:50:40.026 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T19:50:40.026 INFO:teuthology.orchestra.run.smithi079.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T19:50:40.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:40.026 DEBUG:teuthology.orchestra.run.smithi079:> ls /sys/fs/fuse/connections 2024-04-26T19:50:40.036 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:2024-04-26T19:50:40.036+0000 7f0b2be1f480 -1 init, newargv = 0x5562e9e8c590 newargc=15 2024-04-26T19:50:40.037 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:ceph-fuse[42193]: starting ceph client 2024-04-26T19:50:40.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:ceph-fuse[42193]: starting fuse 2024-04-26T19:50:40.051 INFO:teuthology.orchestra.run.smithi079.stdout:43 2024-04-26T19:50:40.052 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo stdin-killer -- python3 -c ' 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> import glob 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> import re 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> import os 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> import subprocess 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> def _find_admin_socket(client_name): 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> files = glob.glob(asok_path) 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> 2024-04-26T19:50:40.052 DEBUG:teuthology.orchestra.run.smithi079:> # Given a non-glob path, it better be there 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> if "*" not in asok_path: 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> assert(len(files) == 1) 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> return files[0] 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> for f in files: 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> contents = proc_f.read() 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> if mountpoint in contents: 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> return f 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> print(_find_admin_socket("client.admin")) 2024-04-26T19:50:40.053 DEBUG:teuthology.orchestra.run.smithi079:> ' 2024-04-26T19:50:40.169 INFO:teuthology.orchestra.run.smithi079.stdout:/var/run/ceph/ceph-client.admin.42193.asok 2024-04-26T19:50:40.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:50:40.179 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.42193.asok 2024-04-26T19:50:40.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:40.179 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.42193.asok status 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: "metadata": { 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_sha1": "bc612ffb9c1d5bc27536bc917a705262a36d4387", 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: "ceph_version": "ceph version 19.0.0-3306-gbc612ffb (bc612ffb9c1d5bc27536bc917a705262a36d4387) squid (dev)", 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: "entity_id": "admin", 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: "hostname": "smithi079", 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: "pid": "42193", 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: "root": "/" 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:50:40.312 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_count": 0, 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "dentry_pinned_count": 0, 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4464, 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "inst": { 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "name": { 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "client", 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "num": 4464 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 728088147 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": { 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "type": "v1", 2024-04-26T19:50:40.313 INFO:teuthology.orchestra.run.smithi079.stdout: "addr": "192.168.0.1:0", 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: "nonce": 728088147 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: "inst_str": "client.4464 192.168.0.1:0/728088147", 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: "addr_str": "192.168.0.1:0/728088147", 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: "inode_count": 1, 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: "mds_epoch": 7, 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch": 25, 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: "osd_epoch_barrier": 0, 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: "blocklisted": false, 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs" 2024-04-26T19:50:40.314 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:50:40.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:40.320 DEBUG:teuthology.orchestra.run.smithi079:> stat --file-system '--printf=%T 2024-04-26T19:50:40.320 DEBUG:teuthology.orchestra.run.smithi079:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-26T19:50:40.336 INFO:teuthology.orchestra.run.smithi079.stdout:fuseblk 2024-04-26T19:50:40.336 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-26T19:50:40.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:40.336 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-26T19:50:40.400 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-26T19:50:40.490 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:50:40.503 INFO:teuthology.orchestra.run.smithi079.stdout:# file: . 2024-04-26T19:50:40.503 INFO:teuthology.orchestra.run.smithi079.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-26T19:50:40.504 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:50:40.509 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:40.510 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:50:40.525 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:40.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.660+0000 7fdec5f7f640 1 Processor -- start 2024-04-26T19:50:40.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.660+0000 7fdec5f7f640 1 -- start start 2024-04-26T19:50:40.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdec5f7f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec00b64f0 0x7fdec0070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdec5f7f640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdec006e800 0x7fdec00b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdec5f7f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdec015f810 con 0x7fdec015ffa0 2024-04-26T19:50:40.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdec5f7f640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fdec015fb10 con 0x7fdec006e800 2024-04-26T19:50:40.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdec5f7f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fdec006f7a0 con 0x7fdec00b64f0 2024-04-26T19:50:40.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdebeffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdec006e800 0x7fdec00b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdebeffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdec006e800 0x7fdec00b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57856/0 (socket says 172.21.15.79:57856) 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdebeffd640 1 -- 172.21.15.79:0/2157254309 learned_addr learned my addr 172.21.15.79:0/2157254309 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdebf7fe640 1 --2- 172.21.15.79:0/2157254309 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec00b64f0 0x7fdec0070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdebeffd640 1 -- 172.21.15.79:0/2157254309 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec00b64f0 msgr2=0x7fdec0070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdebeffd640 1 --2- 172.21.15.79:0/2157254309 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec00b64f0 0x7fdec0070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdebeffd640 1 -- 172.21.15.79:0/2157254309 >> v1:172.21.15.83:6789/0 conn(0x7fdec015ffa0 legacy=0x7fdec0160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdebeffd640 1 -- 172.21.15.79:0/2157254309 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec006ec60 con 0x7fdec006e800 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.661+0000 7fdebf7fe640 1 --2- 172.21.15.79:0/2157254309 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec00b64f0 0x7fdec0070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.662+0000 7fdebeffd640 1 --2- 172.21.15.79:0/2157254309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec006e800 0x7fdec00b68d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009870 tx=0x7fdeb402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92bee1f419062dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.662+0000 7fdebe7fc640 1 -- 172.21.15.79:0/2157254309 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4037070 con 0x7fdec006e800 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.662+0000 7fdebe7fc640 1 -- 172.21.15.79:0/2157254309 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4032440 con 0x7fdec006e800 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.662+0000 7fdebe7fc640 1 -- 172.21.15.79:0/2157254309 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4032810 con 0x7fdec006e800 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.662+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2157254309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec006e800 msgr2=0x7fdec00b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.662+0000 7fdec5f7f640 1 --2- 172.21.15.79:0/2157254309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec006e800 0x7fdec00b68d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009870 tx=0x7fdeb402a210 comp rx=0 tx=0).stop 2024-04-26T19:50:40.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.663+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2157254309 shutdown_connections 2024-04-26T19:50:40.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.663+0000 7fdec5f7f640 1 --2- 172.21.15.79:0/2157254309 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec006e800 0x7fdec00b68d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.663+0000 7fdec5f7f640 1 --2- 172.21.15.79:0/2157254309 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec00b64f0 0x7fdec0070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.663+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2157254309 >> 172.21.15.79:0/2157254309 conn(0x7fdec006c770 msgr2=0x7fdec006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:40.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.663+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2157254309 shutdown_connections 2024-04-26T19:50:40.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.663+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2157254309 wait complete. 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.664+0000 7fdec5f7f640 1 Processor -- start 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.664+0000 7fdec5f7f640 1 -- start start 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdec5f7f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec006e800 0x7fdec01f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdec5f7f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec00b64f0 0x7fdec014f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdec5f7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec015ffa0 0x7fdec01619e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdec5f7f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdec006f1e0 con 0x7fdec00b64f0 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdec5f7f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdec006ef00 con 0x7fdec015ffa0 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdec5f7f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fdec006f4c0 con 0x7fdec006e800 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebeffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec00b64f0 0x7fdec014f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebeffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec00b64f0 0x7fdec014f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57858/0 (socket says 172.21.15.79:57858) 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebeffd640 1 -- 172.21.15.79:0/2022398746 learned_addr learned my addr 172.21.15.79:0/2022398746 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebf7fe640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec006e800 0x7fdec01f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebffff640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec015ffa0 0x7fdec01619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebeffd640 1 -- 172.21.15.79:0/2022398746 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec006e800 msgr2=0x7fdec01f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebeffd640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec006e800 0x7fdec01f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebeffd640 1 -- 172.21.15.79:0/2022398746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec015ffa0 msgr2=0x7fdec01619e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebeffd640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec015ffa0 0x7fdec01619e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebeffd640 1 -- 172.21.15.79:0/2022398746 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec01506b0 con 0x7fdec00b64f0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebffff640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec015ffa0 0x7fdec01619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdebeffd640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec00b64f0 0x7fdec014f6c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009870 tx=0x7fdeb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdea7fff640 1 -- 172.21.15.79:0/2022398746 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4041020 con 0x7fdec00b64f0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdea7fff640 1 -- 172.21.15.79:0/2022398746 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeb4030970 con 0x7fdec00b64f0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec0150990 con 0x7fdec00b64f0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdea7fff640 1 -- 172.21.15.79:0/2022398746 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4037040 con 0x7fdec00b64f0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.665+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec0150f50 con 0x7fdec00b64f0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.666+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec00506c0 con 0x7fdec00b64f0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.667+0000 7fdea7fff640 1 -- 172.21.15.79:0/2022398746 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdeb4044020 con 0x7fdec00b64f0 2024-04-26T19:50:40.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.667+0000 7fdea7fff640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdeac06c8f0 0x7fdeac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.667+0000 7fdea7fff640 1 -- 172.21.15.79:0/2022398746 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdeb40ac7f0 con 0x7fdec00b64f0 2024-04-26T19:50:40.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.667+0000 7fdebf7fe640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdeac06c8f0 0x7fdeac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.668+0000 7fdebf7fe640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdeac06c8f0 0x7fdeac06edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000ab60 tx=0x7fdeb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:40.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.671+0000 7fdea7fff640 1 -- 172.21.15.79:0/2022398746 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdeb404b030 con 0x7fdec00b64f0 2024-04-26T19:50:40.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.845+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdec00ce800 con 0x7fdec00b64f0 2024-04-26T19:50:40.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.846+0000 7fdea7fff640 1 -- 172.21.15.79:0/2022398746 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fdeb407a8e0 con 0x7fdec00b64f0 2024-04-26T19:50:40.847 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:40.847 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:40.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdeac06c8f0 msgr2=0x7fdeac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:40.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdeac06c8f0 0x7fdeac06edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000ab60 tx=0x7fdeb0008040 comp rx=0 tx=0).stop 2024-04-26T19:50:40.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec00b64f0 msgr2=0x7fdec014f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:40.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec00b64f0 0x7fdec014f6c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009870 tx=0x7fdeb40046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 shutdown_connections 2024-04-26T19:50:40.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdeac06c8f0 0x7fdeac06edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdec015ffa0 0x7fdec01619e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdec00b64f0 0x7fdec014f6c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 --2- 172.21.15.79:0/2022398746 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdec006e800 0x7fdec01f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 >> 172.21.15.79:0/2022398746 conn(0x7fdec006c770 msgr2=0x7fdec00cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:40.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 shutdown_connections 2024-04-26T19:50:40.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.848+0000 7fdec5f7f640 1 -- 172.21.15.79:0/2022398746 wait complete. 2024-04-26T19:50:40.860 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:50:40.860 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:50:40.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.959+0000 7f9ad46b0640 1 Processor -- start 2024-04-26T19:50:40.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.959+0000 7f9ad46b0640 1 -- start start 2024-04-26T19:50:40.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.960+0000 7f9ad46b0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9acc142480 0x7f9acc163010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.960+0000 7f9ad46b0640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9acc071f90 0x7f9acc144b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.960+0000 7f9ad46b0640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9acc163e90 con 0x7f9acc1450a0 2024-04-26T19:50:40.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.960+0000 7f9ad46b0640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f9acc1635f0 con 0x7f9acc071f90 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.960+0000 7f9ad46b0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9acc1638d0 con 0x7f9acc142480 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.960+0000 7f9ad1c4b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9acc071f90 0x7f9acc144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.960+0000 7f9ad1c4b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9acc071f90 0x7f9acc144b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57864/0 (socket says 172.21.15.79:57864) 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.960+0000 7f9ad1c4b640 1 -- 172.21.15.79:0/4266852811 learned_addr learned my addr 172.21.15.79:0/4266852811 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.960+0000 7f9ad2c4d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9acc1450a0 0x7f9acc160050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44310/0 (socket says 172.21.15.79:44310) 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.961+0000 7f9ad244c640 1 --2- 172.21.15.79:0/4266852811 >> v2:172.21.15.83:3300/0 conn(0x7f9acc142480 0x7f9acc163010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.961+0000 7f9ad1c4b640 1 -- 172.21.15.79:0/4266852811 >> v2:172.21.15.83:3300/0 conn(0x7f9acc142480 msgr2=0x7f9acc163010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.961+0000 7f9ad1c4b640 1 --2- 172.21.15.79:0/4266852811 >> v2:172.21.15.83:3300/0 conn(0x7f9acc142480 0x7f9acc163010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.961+0000 7f9ad144a640 1 -- 172.21.15.79:0/4266852811 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3709069936 0 0) 0x7f9acc163e90 con 0x7f9acc1450a0 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.961+0000 7f9ad1c4b640 1 -- 172.21.15.79:0/4266852811 >> v1:172.21.15.79:6789/0 conn(0x7f9acc1450a0 legacy=0x7f9acc160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:40.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.961+0000 7f9ad1c4b640 1 -- 172.21.15.79:0/4266852811 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc163bb0 con 0x7f9acc071f90 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.961+0000 7f9ad1c4b640 1 --2- 172.21.15.79:0/4266852811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc071f90 0x7f9acc144b60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9acc072780 tx=0x7f9abc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a428e0a4dec7d2a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad144a640 1 -- 172.21.15.79:0/4266852811 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc037070 con 0x7f9acc071f90 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad144a640 1 -- 172.21.15.79:0/4266852811 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9abc032440 con 0x7f9acc071f90 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad144a640 1 -- 172.21.15.79:0/4266852811 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9abc032810 con 0x7f9acc071f90 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4266852811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc071f90 msgr2=0x7f9acc144b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad46b0640 1 --2- 172.21.15.79:0/4266852811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc071f90 0x7f9acc144b60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9acc072780 tx=0x7f9abc02a270 comp rx=0 tx=0).stop 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4266852811 shutdown_connections 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad46b0640 1 --2- 172.21.15.79:0/4266852811 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc071f90 0x7f9acc144b60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad46b0640 1 --2- 172.21.15.79:0/4266852811 >> v2:172.21.15.83:3300/0 conn(0x7f9acc142480 0x7f9acc163010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4266852811 >> 172.21.15.79:0/4266852811 conn(0x7f9acc0b3ed0 msgr2=0x7f9acc0681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:40.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4266852811 shutdown_connections 2024-04-26T19:50:40.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.962+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4266852811 wait complete. 2024-04-26T19:50:40.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.963+0000 7f9ad46b0640 1 Processor -- start 2024-04-26T19:50:40.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.963+0000 7f9ad46b0640 1 -- start start 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad46b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc071f90 0x7f9acc0b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad46b0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9acc142480 0x7f9acc140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad46b0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc1450a0 0x7f9acc1411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad46b0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc1608e0 con 0x7f9acc1450a0 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad46b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9acc160760 con 0x7f9acc071f90 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad46b0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9acc0d1610 con 0x7f9acc142480 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad2c4d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc1450a0 0x7f9acc1411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad2c4d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc1450a0 0x7f9acc1411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57878/0 (socket says 172.21.15.79:57878) 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad2c4d640 1 -- 172.21.15.79:0/4229674440 learned_addr learned my addr 172.21.15.79:0/4229674440 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad244c640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc071f90 0x7f9acc0b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad1c4b640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9acc142480 0x7f9acc140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad2c4d640 1 -- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9acc142480 msgr2=0x7f9acc140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad2c4d640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9acc142480 0x7f9acc140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad2c4d640 1 -- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc071f90 msgr2=0x7f9acc0b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad2c4d640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc071f90 0x7f9acc0b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad2c4d640 1 -- 172.21.15.79:0/4229674440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9acc142c00 con 0x7f9acc1450a0 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad244c640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc071f90 0x7f9acc0b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.964+0000 7f9ad1c4b640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9acc142480 0x7f9acc140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.965+0000 7f9ad2c4d640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc1450a0 0x7f9acc1411f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400c440 tx=0x7f9ac400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.965+0000 7f9ac2ffd640 1 -- 172.21.15.79:0/4229674440 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40074e0 con 0x7f9acc1450a0 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.965+0000 7f9ac2ffd640 1 -- 172.21.15.79:0/4229674440 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ac40047e0 con 0x7f9acc1450a0 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.965+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9acc142ee0 con 0x7f9acc1450a0 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.965+0000 7f9ac2ffd640 1 -- 172.21.15.79:0/4229674440 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4004bd0 con 0x7f9acc1450a0 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.965+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9acc1434a0 con 0x7f9acc1450a0 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.966+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9acc0506c0 con 0x7f9acc1450a0 2024-04-26T19:50:40.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.966+0000 7f9ac2ffd640 1 -- 172.21.15.79:0/4229674440 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ac4011070 con 0x7f9acc1450a0 2024-04-26T19:50:40.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.966+0000 7f9ac2ffd640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ab006c920 0x7f9ab006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:40.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.967+0000 7f9ac2ffd640 1 -- 172.21.15.79:0/4229674440 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9ac408ba30 con 0x7f9acc1450a0 2024-04-26T19:50:40.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.967+0000 7f9ad244c640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ab006c920 0x7f9ab006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:40.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.967+0000 7f9ad244c640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ab006c920 0x7f9ab006ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8007f50 tx=0x7f9ac80074a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:40.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:40.971+0000 7f9ac2ffd640 1 -- 172.21.15.79:0/4229674440 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9ac4059b20 con 0x7f9acc1450a0 2024-04-26T19:50:41.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.148+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9acc162d80 con 0x7f9acc1450a0 2024-04-26T19:50:41.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.149+0000 7f9ac2ffd640 1 -- 172.21.15.79:0/4229674440 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f9ac4059420 con 0x7f9acc1450a0 2024-04-26T19:50:41.150 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:50:41.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ab006c920 msgr2=0x7f9ab006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ab006c920 0x7f9ab006ede0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8007f50 tx=0x7f9ac80074a0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc1450a0 msgr2=0x7f9acc1411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc1450a0 0x7f9acc1411f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400c440 tx=0x7f9ac400c900 comp rx=0 tx=0).stop 2024-04-26T19:50:41.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 shutdown_connections 2024-04-26T19:50:41.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ab006c920 0x7f9ab006ede0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9acc1450a0 0x7f9acc1411f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9acc142480 0x7f9acc140cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 --2- 172.21.15.79:0/4229674440 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9acc071f90 0x7f9acc0b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 >> 172.21.15.79:0/4229674440 conn(0x7f9acc0b3ed0 msgr2=0x7f9acc0b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:41.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 shutdown_connections 2024-04-26T19:50:41.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.151+0000 7f9ad46b0640 1 -- 172.21.15.79:0/4229674440 wait complete. 2024-04-26T19:50:41.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:50:41.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.261+0000 7faa7de83640 1 Processor -- start 2024-04-26T19:50:41.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.261+0000 7faa7de83640 1 -- start start 2024-04-26T19:50:41.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.262+0000 7faa7de83640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faa78070160 0x7faa780b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.262+0000 7faa7de83640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa7806fa10 con 0x7faa780b1990 2024-04-26T19:50:41.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.262+0000 7faa7de83640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa78072ea0 con 0x7faa780aeeb0 2024-04-26T19:50:41.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.262+0000 7faa7de83640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7faa7806f7f0 con 0x7faa78070160 2024-04-26T19:50:41.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.262+0000 7faa76ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faa78070160 0x7faa780b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.262+0000 7faa77fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7faa780b1990 0x7faa780af290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44322/0 (socket says 172.21.15.79:44322) 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.262+0000 7faa76ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7faa78070160 0x7faa780b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57888/0 (socket says 172.21.15.79:57888) 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.262+0000 7faa77fff640 1 -- 172.21.15.79:0/3058190003 learned_addr learned my addr 172.21.15.79:0/3058190003 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.263+0000 7faa76ffd640 1 -- 172.21.15.79:0/3058190003 >> v1:172.21.15.83:6789/0 conn(0x7faa780aeeb0 legacy=0x7faa7806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.263+0000 7faa76ffd640 1 -- 172.21.15.79:0/3058190003 >> v1:172.21.15.79:6789/0 conn(0x7faa780b1990 legacy=0x7faa780af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.263+0000 7faa76ffd640 1 -- 172.21.15.79:0/3058190003 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa78073180 con 0x7faa78070160 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.263+0000 7faa76ffd640 1 --2- 172.21.15.79:0/3058190003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa78070160 0x7faa780b1450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0097a0 tx=0x7faa6c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be21b923377f7efb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.263+0000 7faa767fc640 1 -- 172.21.15.79:0/3058190003 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c037070 con 0x7faa78070160 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.263+0000 7faa767fc640 1 -- 172.21.15.79:0/3058190003 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6c032440 con 0x7faa78070160 2024-04-26T19:50:41.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.263+0000 7faa767fc640 1 -- 172.21.15.79:0/3058190003 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c032810 con 0x7faa78070160 2024-04-26T19:50:41.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.264+0000 7faa7de83640 1 -- 172.21.15.79:0/3058190003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa78070160 msgr2=0x7faa780b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.264+0000 7faa7de83640 1 --2- 172.21.15.79:0/3058190003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa78070160 0x7faa780b1450 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0097a0 tx=0x7faa6c02a1a0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.264+0000 7faa7de83640 1 -- 172.21.15.79:0/3058190003 shutdown_connections 2024-04-26T19:50:41.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.264+0000 7faa7de83640 1 --2- 172.21.15.79:0/3058190003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa78070160 0x7faa780b1450 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.264+0000 7faa7de83640 1 -- 172.21.15.79:0/3058190003 >> 172.21.15.79:0/3058190003 conn(0x7faa780687e0 msgr2=0x7faa78071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:41.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.264+0000 7faa7de83640 1 -- 172.21.15.79:0/3058190003 shutdown_connections 2024-04-26T19:50:41.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.264+0000 7faa7de83640 1 -- 172.21.15.79:0/3058190003 wait complete. 2024-04-26T19:50:41.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.265+0000 7faa7de83640 1 Processor -- start 2024-04-26T19:50:41.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.265+0000 7faa7de83640 1 -- start start 2024-04-26T19:50:41.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa7de83640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7faa78070160 0x7faa7813bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa7de83640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa780aeeb0 0x7faa781457b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa7de83640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa780b1990 0x7faa7813c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa7de83640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7faa78149380 con 0x7faa780aeeb0 2024-04-26T19:50:41.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa7de83640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7faa78073740 con 0x7faa780b1990 2024-04-26T19:50:41.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa7de83640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7faa780af9a0 con 0x7faa78070160 2024-04-26T19:50:41.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa76ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa780aeeb0 0x7faa781457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa76ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa780aeeb0 0x7faa781457b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57896/0 (socket says 172.21.15.79:57896) 2024-04-26T19:50:41.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa76ffd640 1 -- 172.21.15.79:0/3281405148 learned_addr learned my addr 172.21.15.79:0/3281405148 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:41.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa77fff640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa780b1990 0x7faa7813c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa777fe640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7faa78070160 0x7faa7813bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa76ffd640 1 -- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7faa78070160 msgr2=0x7faa7813bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa76ffd640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7faa78070160 0x7faa7813bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa76ffd640 1 -- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa780b1990 msgr2=0x7faa7813c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa76ffd640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa780b1990 0x7faa7813c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa76ffd640 1 -- 172.21.15.79:0/3281405148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa78156620 con 0x7faa780aeeb0 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa77fff640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa780b1990 0x7faa7813c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.266+0000 7faa777fe640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7faa78070160 0x7faa7813bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.267+0000 7faa76ffd640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa780aeeb0 0x7faa781457b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0098c0 tx=0x7faa6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.267+0000 7faa5ffff640 1 -- 172.21.15.79:0/3281405148 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c037040 con 0x7faa780aeeb0 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.267+0000 7faa5ffff640 1 -- 172.21.15.79:0/3281405148 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa6c030770 con 0x7faa780aeeb0 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.267+0000 7faa5ffff640 1 -- 172.21.15.79:0/3281405148 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c030b40 con 0x7faa780aeeb0 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.267+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa78148330 con 0x7faa780aeeb0 2024-04-26T19:50:41.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.267+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa781488d0 con 0x7faa780aeeb0 2024-04-26T19:50:41.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.268+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa780506c0 con 0x7faa780aeeb0 2024-04-26T19:50:41.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.268+0000 7faa5ffff640 1 -- 172.21.15.79:0/3281405148 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7faa6c044020 con 0x7faa780aeeb0 2024-04-26T19:50:41.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.269+0000 7faa5ffff640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7faa6406c8f0 0x7faa6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.269+0000 7faa5ffff640 1 -- 172.21.15.79:0/3281405148 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faa6c0ac390 con 0x7faa780aeeb0 2024-04-26T19:50:41.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.269+0000 7faa777fe640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7faa6406c8f0 0x7faa6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.269+0000 7faa777fe640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7faa6406c8f0 0x7faa6406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa68001700 tx=0x7faa68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:41.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.273+0000 7faa5ffff640 1 -- 172.21.15.79:0/3281405148 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7faa78156620 con 0x7faa780aeeb0 2024-04-26T19:50:41.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.447+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faa7806a800 con 0x7faa780aeeb0 2024-04-26T19:50:41.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.448+0000 7faa5ffff640 1 -- 172.21.15.79:0/3281405148 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7faa6c032e20 con 0x7faa780aeeb0 2024-04-26T19:50:41.448 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:50:41.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7faa6406c8f0 msgr2=0x7faa6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7faa6406c8f0 0x7faa6406edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faa68001700 tx=0x7faa68009040 comp rx=0 tx=0).stop 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa780aeeb0 msgr2=0x7faa781457b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa780aeeb0 0x7faa781457b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0098c0 tx=0x7faa6c0046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 shutdown_connections 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7faa6406c8f0 0x7faa6406edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7faa780b1990 0x7faa7813c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7faa780aeeb0 0x7faa781457b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 --2- 172.21.15.79:0/3281405148 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7faa78070160 0x7faa7813bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 >> 172.21.15.79:0/3281405148 conn(0x7faa780687e0 msgr2=0x7faa78144b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 shutdown_connections 2024-04-26T19:50:41.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.450+0000 7faa7de83640 1 -- 172.21.15.79:0/3281405148 wait complete. 2024-04-26T19:50:41.461 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:50:41.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.559+0000 7fecc4adc640 1 Processor -- start 2024-04-26T19:50:41.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.559+0000 7fecc4adc640 1 -- start start 2024-04-26T19:50:41.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.560+0000 7fecc4adc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fecc0070160 0x7fecc00b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.560+0000 7fecc4adc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fecc00b1990 0x7fecc00af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.560+0000 7fecc4adc640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc006fa10 con 0x7fecc00aeeb0 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.560+0000 7fecc4adc640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fecc006f7f0 con 0x7fecc0070160 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.560+0000 7fecc4adc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fecc0072ea0 con 0x7fecc00b1990 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.560+0000 7fecbdd9b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fecc0070160 0x7fecc00b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.560+0000 7fecbe59c640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fecc00aeeb0 0x7fecc006f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44328/0 (socket says 172.21.15.79:44328) 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.560+0000 7fecbdd9b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fecc0070160 0x7fecc00b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57898/0 (socket says 172.21.15.79:57898) 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.560+0000 7fecbe59c640 1 -- 172.21.15.79:0/584922980 learned_addr learned my addr 172.21.15.79:0/584922980 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbed9d640 1 --2- 172.21.15.79:0/584922980 >> v2:172.21.15.83:3300/0 conn(0x7fecc00b1990 0x7fecc00af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbdd9b640 1 -- 172.21.15.79:0/584922980 >> v2:172.21.15.83:3300/0 conn(0x7fecc00b1990 msgr2=0x7fecc00af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbdd9b640 1 --2- 172.21.15.79:0/584922980 >> v2:172.21.15.83:3300/0 conn(0x7fecc00b1990 0x7fecc00af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbdd9b640 1 -- 172.21.15.79:0/584922980 >> v1:172.21.15.79:6789/0 conn(0x7fecc00aeeb0 legacy=0x7fecc006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbdd9b640 1 -- 172.21.15.79:0/584922980 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc0073180 con 0x7fecc0070160 2024-04-26T19:50:41.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbd59a640 1 -- 172.21.15.79:0/584922980 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034218622 0 0) 0x7fecc006fa10 con 0x7fecc00aeeb0 2024-04-26T19:50:41.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbdd9b640 1 --2- 172.21.15.79:0/584922980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc0070160 0x7fecc00b1450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fecc00b3f70 tx=0x7fecb402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd46126551aeb54d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbd59a640 1 -- 172.21.15.79:0/584922980 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4037070 con 0x7fecc0070160 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbd59a640 1 -- 172.21.15.79:0/584922980 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb4032440 con 0x7fecc0070160 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.561+0000 7fecbd59a640 1 -- 172.21.15.79:0/584922980 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb402ae60 con 0x7fecc0070160 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.562+0000 7fecc4adc640 1 -- 172.21.15.79:0/584922980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc0070160 msgr2=0x7fecc00b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.562+0000 7fecc4adc640 1 --2- 172.21.15.79:0/584922980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc0070160 0x7fecc00b1450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fecc00b3f70 tx=0x7fecb402a110 comp rx=0 tx=0).stop 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.562+0000 7fecc4adc640 1 -- 172.21.15.79:0/584922980 shutdown_connections 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.562+0000 7fecc4adc640 1 --2- 172.21.15.79:0/584922980 >> v2:172.21.15.83:3300/0 conn(0x7fecc00b1990 0x7fecc00af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.562+0000 7fecc4adc640 1 --2- 172.21.15.79:0/584922980 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc0070160 0x7fecc00b1450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.562+0000 7fecc4adc640 1 -- 172.21.15.79:0/584922980 >> 172.21.15.79:0/584922980 conn(0x7fecc00687e0 msgr2=0x7fecc0071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.562+0000 7fecc4adc640 1 -- 172.21.15.79:0/584922980 shutdown_connections 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.562+0000 7fecc4adc640 1 -- 172.21.15.79:0/584922980 wait complete. 2024-04-26T19:50:41.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.563+0000 7fecc4adc640 1 Processor -- start 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.563+0000 7fecc4adc640 1 -- start start 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecc4adc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fecc0070160 0x7fecc01f5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecc4adc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc00aeeb0 0x7fecc0162130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecc4adc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc00b1990 0x7fecc0162670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecc4adc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0149380 con 0x7fecc00b1990 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecc4adc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0073740 con 0x7fecc00aeeb0 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecc4adc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fecc00af7d0 con 0x7fecc0070160 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc00b1990 0x7fecc0162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc00b1990 0x7fecc0162670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57914/0 (socket says 172.21.15.79:57914) 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbed9d640 1 -- 172.21.15.79:0/4260934770 learned_addr learned my addr 172.21.15.79:0/4260934770 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbe59c640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fecc0070160 0x7fecc01f5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbdd9b640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc00aeeb0 0x7fecc0162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbed9d640 1 -- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fecc0070160 msgr2=0x7fecc01f5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbed9d640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fecc0070160 0x7fecc01f5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbed9d640 1 -- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc00aeeb0 msgr2=0x7fecc0162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbed9d640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc00aeeb0 0x7fecc0162130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbed9d640 1 -- 172.21.15.79:0/4260934770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc0142630 con 0x7fecc00b1990 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbe59c640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fecc0070160 0x7fecc01f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.564+0000 7fecbdd9b640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc00aeeb0 0x7fecc0162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.565+0000 7fecbed9d640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc00b1990 0x7fecc0162670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fecb800d600 tx=0x7fecb800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.565+0000 7fecaeffd640 1 -- 172.21.15.79:0/4260934770 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80125f0 con 0x7fecc00b1990 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.565+0000 7fecaeffd640 1 -- 172.21.15.79:0/4260934770 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb8019070 con 0x7fecc00b1990 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.565+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc01f86e0 con 0x7fecc00b1990 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.565+0000 7fecaeffd640 1 -- 172.21.15.79:0/4260934770 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80129f0 con 0x7fecc00b1990 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.565+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc01f8ca0 con 0x7fecc00b1990 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.566+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc00506c0 con 0x7fecc00b1990 2024-04-26T19:50:41.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.566+0000 7fecaeffd640 1 -- 172.21.15.79:0/4260934770 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fecb8012b90 con 0x7fecc00b1990 2024-04-26T19:50:41.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.567+0000 7fecaeffd640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feca006c920 0x7feca006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.567+0000 7fecbe59c640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feca006c920 0x7feca006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.567+0000 7fecaeffd640 1 -- 172.21.15.79:0/4260934770 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fecb808c220 con 0x7fecc00b1990 2024-04-26T19:50:41.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.567+0000 7fecbe59c640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feca006c920 0x7feca006ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fecb00012e0 tx=0x7fecb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:41.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.571+0000 7fecaeffd640 1 -- 172.21.15.79:0/4260934770 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fecb805a320 con 0x7fecc00b1990 2024-04-26T19:50:41.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.746+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fecc006a800 con 0x7fecc00b1990 2024-04-26T19:50:41.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.747+0000 7fecaeffd640 1 -- 172.21.15.79:0/4260934770 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fecb8059c20 con 0x7fecc00b1990 2024-04-26T19:50:41.747 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:50:41.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feca006c920 msgr2=0x7feca006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feca006c920 0x7feca006ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fecb00012e0 tx=0x7fecb0009040 comp rx=0 tx=0).stop 2024-04-26T19:50:41.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc00b1990 msgr2=0x7fecc0162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc00b1990 0x7fecc0162670 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fecb800d600 tx=0x7fecb800dac0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 shutdown_connections 2024-04-26T19:50:41.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feca006c920 0x7feca006ede0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fecc00b1990 0x7fecc0162670 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fecc00aeeb0 0x7fecc0162130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 --2- 172.21.15.79:0/4260934770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fecc0070160 0x7fecc01f5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 >> 172.21.15.79:0/4260934770 conn(0x7fecc00687e0 msgr2=0x7fecc015f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:41.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 shutdown_connections 2024-04-26T19:50:41.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.749+0000 7fecc4adc640 1 -- 172.21.15.79:0/4260934770 wait complete. 2024-04-26T19:50:41.760 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:50:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.859+0000 7f428db12640 1 Processor -- start 2024-04-26T19:50:41.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.859+0000 7f428db12640 1 -- start start 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.860+0000 7f428db12640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f428815c1e0 0x7f4288073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.860+0000 7f428db12640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f428806f920 0x7f428806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.860+0000 7f428db12640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4288072620 con 0x7f42880718f0 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.860+0000 7f428db12640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f42880702c0 con 0x7f428806f920 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.860+0000 7f428db12640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f42880730d0 con 0x7f428815c1e0 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.860+0000 7f4286ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f428806f920 0x7f428806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.860+0000 7f4286ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f428806f920 0x7f428806fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57924/0 (socket says 172.21.15.79:57924) 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.860+0000 7f4286ffd640 1 -- 172.21.15.79:0/4068740765 learned_addr learned my addr 172.21.15.79:0/4068740765 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.860+0000 7f42877fe640 1 --2- 172.21.15.79:0/4068740765 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f428815c1e0 0x7f4288073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.861+0000 7f4286ffd640 1 -- 172.21.15.79:0/4068740765 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f428815c1e0 msgr2=0x7f4288073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.861+0000 7f4286ffd640 1 --2- 172.21.15.79:0/4068740765 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f428815c1e0 0x7f4288073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.862 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.861+0000 7f4286ffd640 1 -- 172.21.15.79:0/4068740765 >> v1:172.21.15.83:6789/0 conn(0x7f42880718f0 legacy=0x7f4288071d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.861+0000 7f4286ffd640 1 -- 172.21.15.79:0/4068740765 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42880733b0 con 0x7f428806f920 2024-04-26T19:50:41.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.861+0000 7f4286ffd640 1 --2- 172.21.15.79:0/4068740765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 0x7f428806fd80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f427c009870 tx=0x7f427c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfad35e5ab005a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:41.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.861+0000 7f42867fc640 1 -- 172.21.15.79:0/4068740765 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c037070 con 0x7f428806f920 2024-04-26T19:50:41.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.861+0000 7f42867fc640 1 -- 172.21.15.79:0/4068740765 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427c032440 con 0x7f428806f920 2024-04-26T19:50:41.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.861+0000 7f42867fc640 1 -- 172.21.15.79:0/4068740765 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c032810 con 0x7f428806f920 2024-04-26T19:50:41.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.862+0000 7f428db12640 1 -- 172.21.15.79:0/4068740765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 msgr2=0x7f428806fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.862+0000 7f428db12640 1 --2- 172.21.15.79:0/4068740765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 0x7f428806fd80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f427c009870 tx=0x7f427c02a130 comp rx=0 tx=0).stop 2024-04-26T19:50:41.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.862+0000 7f428db12640 1 -- 172.21.15.79:0/4068740765 shutdown_connections 2024-04-26T19:50:41.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.862+0000 7f428db12640 1 --2- 172.21.15.79:0/4068740765 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 0x7f428806fd80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.862+0000 7f428db12640 1 --2- 172.21.15.79:0/4068740765 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f428815c1e0 0x7f4288073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.862+0000 7f428db12640 1 -- 172.21.15.79:0/4068740765 >> 172.21.15.79:0/4068740765 conn(0x7f42880b2540 msgr2=0x7f42880b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:41.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.862+0000 7f428db12640 1 -- 172.21.15.79:0/4068740765 shutdown_connections 2024-04-26T19:50:41.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.862+0000 7f428db12640 1 -- 172.21.15.79:0/4068740765 wait complete. 2024-04-26T19:50:41.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.863+0000 7f428db12640 1 Processor -- start 2024-04-26T19:50:41.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.863+0000 7f428db12640 1 -- start start 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f428db12640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 0x7f4288142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f428db12640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f42880718f0 0x7f428806e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f428db12640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428815c1e0 0x7f428806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f428db12640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f42880cde10 con 0x7f428806f920 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f428db12640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f42880cdc90 con 0x7f428815c1e0 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f428db12640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f42880cdf90 con 0x7f42880718f0 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f42877fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 0x7f4288142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f42877fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 0x7f4288142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57932/0 (socket says 172.21.15.79:57932) 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f42877fe640 1 -- 172.21.15.79:0/235874781 learned_addr learned my addr 172.21.15.79:0/235874781 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f4286ffd640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f42880718f0 0x7f428806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f4287fff640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428815c1e0 0x7f428806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f42877fe640 1 -- 172.21.15.79:0/235874781 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f42880718f0 msgr2=0x7f428806e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f42877fe640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f42880718f0 0x7f428806e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f42877fe640 1 -- 172.21.15.79:0/235874781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428815c1e0 msgr2=0x7f428806ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f42877fe640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428815c1e0 0x7f428806ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f42877fe640 1 -- 172.21.15.79:0/235874781 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4288143140 con 0x7f428806f920 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.864+0000 7f42877fe640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 0x7f4288142c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f427800e8a0 tx=0x7f427800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.865+0000 7f426ffff640 1 -- 172.21.15.79:0/235874781 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278017070 con 0x7f428806f920 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.865+0000 7f426ffff640 1 -- 172.21.15.79:0/235874781 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427800c940 con 0x7f428806f920 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.865+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428815a130 con 0x7f428806f920 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.865+0000 7f426ffff640 1 -- 172.21.15.79:0/235874781 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427800cd30 con 0x7f428806f920 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.865+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f428815a6f0 con 0x7f428806f920 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.866+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42880506c0 con 0x7f428806f920 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.866+0000 7f426ffff640 1 -- 172.21.15.79:0/235874781 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f427801a070 con 0x7f428806f920 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.866+0000 7f426ffff640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427406c8f0 0x7f427406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.866+0000 7f426ffff640 1 -- 172.21.15.79:0/235874781 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f427808bce0 con 0x7f428806f920 2024-04-26T19:50:41.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.866+0000 7f4286ffd640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427406c8f0 0x7f427406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:41.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.867+0000 7f4286ffd640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427406c8f0 0x7f427406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f427c034000 tx=0x7f427c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:41.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:41.871+0000 7f426ffff640 1 -- 172.21.15.79:0/235874781 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4278059dd0 con 0x7f428806f920 2024-04-26T19:50:42.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.047+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f42880cda70 con 0x7f428806f920 2024-04-26T19:50:42.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.047+0000 7f426ffff640 1 -- 172.21.15.79:0/235874781 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f42780596d0 con 0x7f428806f920 2024-04-26T19:50:42.048 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:50:42.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427406c8f0 msgr2=0x7f427406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427406c8f0 0x7f427406edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f427c034000 tx=0x7f427c02e000 comp rx=0 tx=0).stop 2024-04-26T19:50:42.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 msgr2=0x7f4288142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 0x7f4288142c00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f427800e8a0 tx=0x7f427800ed60 comp rx=0 tx=0).stop 2024-04-26T19:50:42.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 shutdown_connections 2024-04-26T19:50:42.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427406c8f0 0x7f427406edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f428815c1e0 0x7f428806ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f42880718f0 0x7f428806e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 --2- 172.21.15.79:0/235874781 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f428806f920 0x7f4288142c00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.049+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 >> 172.21.15.79:0/235874781 conn(0x7f42880b2540 msgr2=0x7f4288073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:42.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.050+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 shutdown_connections 2024-04-26T19:50:42.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.050+0000 7f428db12640 1 -- 172.21.15.79:0/235874781 wait complete. 2024-04-26T19:50:42.061 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T19:50:42.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.158+0000 7f0fd07da640 1 Processor -- start 2024-04-26T19:50:42.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.158+0000 7f0fd07da640 1 -- start start 2024-04-26T19:50:42.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.159+0000 7f0fd07da640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc80b7ce0 0x7f0fc80b8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.159+0000 7f0fd07da640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc80d0d90 con 0x7f0fc80d1210 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.159+0000 7f0fd07da640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc80cfcf0 con 0x7f0fc8154220 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.159+0000 7f0fd07da640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc80b8680 con 0x7f0fc80b7ce0 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.159+0000 7f0fced77640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0fc80d1210 0x7f0fc80d16a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44334/0 (socket says 172.21.15.79:44334) 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.159+0000 7f0fced77640 1 -- 172.21.15.79:0/3727022632 learned_addr learned my addr 172.21.15.79:0/3727022632 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.159+0000 7f0fcdd75640 1 --2- 172.21.15.79:0/3727022632 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc80b7ce0 0x7f0fc80b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.160+0000 7f0fcd574640 1 -- 172.21.15.79:0/3727022632 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 804846076 0 0) 0x7f0fc80d0d90 con 0x7f0fc80d1210 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.160+0000 7f0fcd574640 1 -- 172.21.15.79:0/3727022632 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fb0003520 con 0x7f0fc80d1210 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.160+0000 7f0fcd574640 1 -- 172.21.15.79:0/3727022632 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 619163215 0 0) 0x7f0fc80cfcf0 con 0x7f0fc8154220 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.160+0000 7f0fcd574640 1 -- 172.21.15.79:0/3727022632 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fc80d0d90 con 0x7f0fc8154220 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.160+0000 7f0fcdd75640 1 -- 172.21.15.79:0/3727022632 >> v1:172.21.15.83:6789/0 conn(0x7f0fc8154220 legacy=0x7f0fc80c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.160+0000 7f0fcdd75640 1 -- 172.21.15.79:0/3727022632 >> v1:172.21.15.79:6789/0 conn(0x7f0fc80d1210 legacy=0x7f0fc80d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.160+0000 7f0fcdd75640 1 -- 172.21.15.79:0/3727022632 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc80cffd0 con 0x7f0fc80b7ce0 2024-04-26T19:50:42.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fcdd75640 1 --2- 172.21.15.79:0/3727022632 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc80b7ce0 0x7f0fc80b8140 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0097d0 tx=0x7f0fbc02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c16393f05b7fea3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:42.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fcd574640 1 -- 172.21.15.79:0/3727022632 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc037070 con 0x7f0fc80b7ce0 2024-04-26T19:50:42.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fcd574640 1 -- 172.21.15.79:0/3727022632 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fbc032440 con 0x7f0fc80b7ce0 2024-04-26T19:50:42.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fcd574640 1 -- 172.21.15.79:0/3727022632 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0327f0 con 0x7f0fc80b7ce0 2024-04-26T19:50:42.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fd07da640 1 -- 172.21.15.79:0/3727022632 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc80b7ce0 msgr2=0x7f0fc80b8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fd07da640 1 --2- 172.21.15.79:0/3727022632 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc80b7ce0 0x7f0fc80b8140 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0097d0 tx=0x7f0fbc02a210 comp rx=0 tx=0).stop 2024-04-26T19:50:42.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fd07da640 1 -- 172.21.15.79:0/3727022632 shutdown_connections 2024-04-26T19:50:42.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fd07da640 1 --2- 172.21.15.79:0/3727022632 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc80b7ce0 0x7f0fc80b8140 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fd07da640 1 -- 172.21.15.79:0/3727022632 >> 172.21.15.79:0/3727022632 conn(0x7f0fc80741a0 msgr2=0x7f0fc80b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:42.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fd07da640 1 -- 172.21.15.79:0/3727022632 shutdown_connections 2024-04-26T19:50:42.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.161+0000 7f0fd07da640 1 -- 172.21.15.79:0/3727022632 wait complete. 2024-04-26T19:50:42.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.162+0000 7f0fd07da640 1 Processor -- start 2024-04-26T19:50:42.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.162+0000 7f0fd07da640 1 -- start start 2024-04-26T19:50:42.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fd07da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fc80b7ce0 0x7f0fc8144ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fd07da640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc80d1210 0x7f0fc814cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fd07da640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc8154220 0x7f0fc815c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fd07da640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc80d0870 con 0x7f0fc80d1210 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fd07da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc80d0590 con 0x7f0fc80b7ce0 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fd07da640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc8154600 con 0x7f0fc8154220 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fcdd75640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc80d1210 0x7f0fc814cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fcdd75640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc80d1210 0x7f0fc814cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57946/0 (socket says 172.21.15.79:57946) 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fcdd75640 1 -- 172.21.15.79:0/1365151457 learned_addr learned my addr 172.21.15.79:0/1365151457 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fced77640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc8154220 0x7f0fc815c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.163+0000 7f0fce576640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fc80b7ce0 0x7f0fc8144ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fcdd75640 1 -- 172.21.15.79:0/1365151457 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc8154220 msgr2=0x7f0fc815c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fcdd75640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc8154220 0x7f0fc815c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fcdd75640 1 -- 172.21.15.79:0/1365151457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fc80b7ce0 msgr2=0x7f0fc8144ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fcdd75640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fc80b7ce0 0x7f0fc8144ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fcdd75640 1 -- 172.21.15.79:0/1365151457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc8155cf0 con 0x7f0fc80d1210 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fcdd75640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc80d1210 0x7f0fc814cfb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc02a250 tx=0x7f0fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:42.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fc6ffd640 1 -- 172.21.15.79:0/1365151457 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc041020 con 0x7f0fc80d1210 2024-04-26T19:50:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fc6ffd640 1 -- 172.21.15.79:0/1365151457 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fbc03d6e0 con 0x7f0fc80d1210 2024-04-26T19:50:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc8155fd0 con 0x7f0fc80d1210 2024-04-26T19:50:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fc6ffd640 1 -- 172.21.15.79:0/1365151457 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc037040 con 0x7f0fc80d1210 2024-04-26T19:50:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.164+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc8156590 con 0x7f0fc80d1210 2024-04-26T19:50:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.165+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fc80506c0 con 0x7f0fc80d1210 2024-04-26T19:50:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.165+0000 7f0fc6ffd640 1 -- 172.21.15.79:0/1365151457 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0fbc044020 con 0x7f0fc80d1210 2024-04-26T19:50:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.166+0000 7f0fc6ffd640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0fb406c8f0 0x7f0fb406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.166+0000 7f0fc6ffd640 1 -- 172.21.15.79:0/1365151457 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0fbc0ac4f0 con 0x7f0fc80d1210 2024-04-26T19:50:42.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.166+0000 7f0fce576640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0fb406c8f0 0x7f0fb406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.166+0000 7f0fce576640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0fb406c8f0 0x7f0fb406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8007fd0 tx=0x7f0fb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:42.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.170+0000 7f0fc6ffd640 1 -- 172.21.15.79:0/1365151457 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0fbc04b030 con 0x7f0fc80d1210 2024-04-26T19:50:42.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.358+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0fc80d0ff0 con 0x7f0fc80d1210 2024-04-26T19:50:42.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.359+0000 7f0fc6ffd640 1 -- 172.21.15.79:0/1365151457 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f0fbc079fa0 con 0x7f0fc80d1210 2024-04-26T19:50:42.360 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2024-04-26T19:50:42.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0fb406c8f0 msgr2=0x7f0fb406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0fb406c8f0 0x7f0fb406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8007fd0 tx=0x7f0fb8009040 comp rx=0 tx=0).stop 2024-04-26T19:50:42.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc80d1210 msgr2=0x7f0fc814cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc80d1210 0x7f0fc814cfb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc02a250 tx=0x7f0fbc0046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 shutdown_connections 2024-04-26T19:50:42.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0fb406c8f0 0x7f0fb406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fc8154220 0x7f0fc815c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fc80d1210 0x7f0fc814cfb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 --2- 172.21.15.79:0/1365151457 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fc80b7ce0 0x7f0fc8144ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 >> 172.21.15.79:0/1365151457 conn(0x7f0fc80741a0 msgr2=0x7f0fc806e560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:42.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 shutdown_connections 2024-04-26T19:50:42.363 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.361+0000 7f0fd07da640 1 -- 172.21.15.79:0/1365151457 wait complete. 2024-04-26T19:50:42.373 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:50:42.373 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:50:42.388 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.523+0000 7fcb0c269640 1 Processor -- start 2024-04-26T19:50:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.523+0000 7fcb0c269640 1 -- start start 2024-04-26T19:50:42.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.524+0000 7fcb0c269640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb04148fd0 0x7fcb040d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.524+0000 7fcb0c269640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb040b1290 con 0x7fcb0406ff00 2024-04-26T19:50:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.524+0000 7fcb0c269640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb040d4090 con 0x7fcb0406fb20 2024-04-26T19:50:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.524+0000 7fcb0c269640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fcb040d4370 con 0x7fcb04148fd0 2024-04-26T19:50:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.524+0000 7fcb0a806640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb04148fd0 0x7fcb040d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.524+0000 7fcb0a806640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb04148fd0 0x7fcb040d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57962/0 (socket says 172.21.15.79:57962) 2024-04-26T19:50:42.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.524+0000 7fcb0a806640 1 -- 172.21.15.79:0/859371338 learned_addr learned my addr 172.21.15.79:0/859371338 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.525+0000 7fcb0a806640 1 -- 172.21.15.79:0/859371338 >> v1:172.21.15.83:6789/0 conn(0x7fcb0406fb20 legacy=0x7fcb041488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.525+0000 7fcb0a806640 1 -- 172.21.15.79:0/859371338 >> v1:172.21.15.79:6789/0 conn(0x7fcb0406ff00 legacy=0x7fcb040aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.525+0000 7fcb09003640 1 -- 172.21.15.79:0/859371338 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523217913 0 0) 0x7fcb040b1290 con 0x7fcb0406ff00 2024-04-26T19:50:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.525+0000 7fcb0a806640 1 -- 172.21.15.79:0/859371338 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb040d4650 con 0x7fcb04148fd0 2024-04-26T19:50:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.525+0000 7fcb0a806640 1 --2- 172.21.15.79:0/859371338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb04148fd0 0x7fcb040d2cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098d0 tx=0x7fcafc02a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ef1125c2e272a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.525+0000 7fcb09003640 1 -- 172.21.15.79:0/859371338 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc037070 con 0x7fcb04148fd0 2024-04-26T19:50:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.525+0000 7fcb09003640 1 -- 172.21.15.79:0/859371338 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcafc032640 con 0x7fcb04148fd0 2024-04-26T19:50:42.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.525+0000 7fcb09003640 1 -- 172.21.15.79:0/859371338 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcafc032a10 con 0x7fcb04148fd0 2024-04-26T19:50:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.526+0000 7fcb0c269640 1 -- 172.21.15.79:0/859371338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb04148fd0 msgr2=0x7fcb040d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.526+0000 7fcb0c269640 1 --2- 172.21.15.79:0/859371338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb04148fd0 0x7fcb040d2cd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0098d0 tx=0x7fcafc02a400 comp rx=0 tx=0).stop 2024-04-26T19:50:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.526+0000 7fcb0c269640 1 -- 172.21.15.79:0/859371338 shutdown_connections 2024-04-26T19:50:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.526+0000 7fcb0c269640 1 --2- 172.21.15.79:0/859371338 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb04148fd0 0x7fcb040d2cd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.526+0000 7fcb0c269640 1 -- 172.21.15.79:0/859371338 >> 172.21.15.79:0/859371338 conn(0x7fcb04071c30 msgr2=0x7fcb0406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.526+0000 7fcb0c269640 1 -- 172.21.15.79:0/859371338 shutdown_connections 2024-04-26T19:50:42.528 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.526+0000 7fcb0c269640 1 -- 172.21.15.79:0/859371338 wait complete. 2024-04-26T19:50:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.527+0000 7fcb0c269640 1 Processor -- start 2024-04-26T19:50:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.527+0000 7fcb0c269640 1 -- start start 2024-04-26T19:50:42.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb0c269640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb0406fb20 0x7fcb04141890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb0c269640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb0406ff00 0x7fcb04142ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb0c269640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb04148fd0 0x7fcb04147690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb0c269640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040d3920 con 0x7fcb0406ff00 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb0c269640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040d4c10 con 0x7fcb04148fd0 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb0c269640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fcb040d3aa0 con 0x7fcb0406fb20 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb09804640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb0406ff00 0x7fcb04142ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb09804640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb0406ff00 0x7fcb04142ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57972/0 (socket says 172.21.15.79:57972) 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb09804640 1 -- 172.21.15.79:0/3577602456 learned_addr learned my addr 172.21.15.79:0/3577602456 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb0a005640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb0406fb20 0x7fcb04141890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb0a806640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb04148fd0 0x7fcb04147690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb09804640 1 -- 172.21.15.79:0/3577602456 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb0406fb20 msgr2=0x7fcb04141890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb09804640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb0406fb20 0x7fcb04141890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb09804640 1 -- 172.21.15.79:0/3577602456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb04148fd0 msgr2=0x7fcb04147690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb09804640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb04148fd0 0x7fcb04147690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb09804640 1 -- 172.21.15.79:0/3577602456 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb04143db0 con 0x7fcb0406ff00 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb0a806640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb04148fd0 0x7fcb04147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.528+0000 7fcb09804640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb0406ff00 0x7fcb04142ba0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c460 tx=0x7fcaf400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.529+0000 7fcafaffd640 1 -- 172.21.15.79:0/3577602456 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4005480 con 0x7fcb0406ff00 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.529+0000 7fcafaffd640 1 -- 172.21.15.79:0/3577602456 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcaf40035f0 con 0x7fcb0406ff00 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.529+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0415ab40 con 0x7fcb0406ff00 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.529+0000 7fcafaffd640 1 -- 172.21.15.79:0/3577602456 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaf4003a00 con 0x7fcb0406ff00 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.529+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb041437f0 con 0x7fcb0406ff00 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.530+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb0415a970 con 0x7fcb0406ff00 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.530+0000 7fcafaffd640 1 -- 172.21.15.79:0/3577602456 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcaf4003ba0 con 0x7fcb0406ff00 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.530+0000 7fcafaffd640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcae806ca90 0x7fcae806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.530+0000 7fcafaffd640 1 -- 172.21.15.79:0/3577602456 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcaf408ba40 con 0x7fcb0406ff00 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.530+0000 7fcb0a005640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcae806ca90 0x7fcae806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.531+0000 7fcb0a005640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcae806ca90 0x7fcae806ef50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb041f9b60 tx=0x7fcb00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:42.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.535+0000 7fcafaffd640 1 -- 172.21.15.79:0/3577602456 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcaf4059b30 con 0x7fcb0406ff00 2024-04-26T19:50:42.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.709+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb040506c0 con 0x7fcb0406ff00 2024-04-26T19:50:42.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.710+0000 7fcafaffd640 1 -- 172.21.15.79:0/3577602456 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fcaf408be20 con 0x7fcb0406ff00 2024-04-26T19:50:42.711 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:42.711 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:42.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcae806ca90 msgr2=0x7fcae806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcae806ca90 0x7fcae806ef50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcb041f9b60 tx=0x7fcb00009040 comp rx=0 tx=0).stop 2024-04-26T19:50:42.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb0406ff00 msgr2=0x7fcb04142ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb0406ff00 0x7fcb04142ba0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400c460 tx=0x7fcaf400c920 comp rx=0 tx=0).stop 2024-04-26T19:50:42.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 shutdown_connections 2024-04-26T19:50:42.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcae806ca90 0x7fcae806ef50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb04148fd0 0x7fcb04147690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb0406ff00 0x7fcb04142ba0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 --2- 172.21.15.79:0/3577602456 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb0406fb20 0x7fcb04141890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 >> 172.21.15.79:0/3577602456 conn(0x7fcb04071c30 msgr2=0x7fcb04072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:42.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 shutdown_connections 2024-04-26T19:50:42.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.712+0000 7fcb0c269640 1 -- 172.21.15.79:0/3577602456 wait complete. 2024-04-26T19:50:42.724 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:50:42.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:50:42.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.823+0000 7fd8e507b640 1 Processor -- start 2024-04-26T19:50:42.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.823+0000 7fd8e507b640 1 -- start start 2024-04-26T19:50:42.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.823+0000 7fd8e507b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8e0148f30 0x7fd8e00b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.823+0000 7fd8e507b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e00b7620 0x7fd8e00b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.823+0000 7fd8e507b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8e00b7e60 con 0x7fd8e006c350 2024-04-26T19:50:42.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.823+0000 7fd8e507b640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd8e00b8140 con 0x7fd8e0148f30 2024-04-26T19:50:42.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.823+0000 7fd8e507b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e00b8420 con 0x7fd8e00b7620 2024-04-26T19:50:42.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.824+0000 7fd8ded9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8e0148f30 0x7fd8e00b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.824+0000 7fd8ded9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8e0148f30 0x7fd8e00b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57980/0 (socket says 172.21.15.79:57980) 2024-04-26T19:50:42.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.824+0000 7fd8ded9d640 1 -- 172.21.15.79:0/528980421 learned_addr learned my addr 172.21.15.79:0/528980421 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.824+0000 7fd8d7fff640 1 --2- 172.21.15.79:0/528980421 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e00b7620 0x7fd8e00b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.824+0000 7fd8ded9d640 1 -- 172.21.15.79:0/528980421 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e00b7620 msgr2=0x7fd8e00b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.824+0000 7fd8ded9d640 1 --2- 172.21.15.79:0/528980421 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e00b7620 0x7fd8e00b8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.824+0000 7fd8ded9d640 1 -- 172.21.15.79:0/528980421 >> v1:172.21.15.83:6789/0 conn(0x7fd8e006c350 legacy=0x7fd8e015fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.824+0000 7fd8ded9d640 1 -- 172.21.15.79:0/528980421 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e00b8700 con 0x7fd8e0148f30 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8ded9d640 1 --2- 172.21.15.79:0/528980421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e0148f30 0x7fd8e00b6630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009870 tx=0x7fd8d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04ed35d6cf505a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8de59c640 1 -- 172.21.15.79:0/528980421 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0037070 con 0x7fd8e0148f30 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8de59c640 1 -- 172.21.15.79:0/528980421 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8d0032440 con 0x7fd8e0148f30 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8de59c640 1 -- 172.21.15.79:0/528980421 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8d0032810 con 0x7fd8e0148f30 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8e507b640 1 -- 172.21.15.79:0/528980421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e0148f30 msgr2=0x7fd8e00b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8e507b640 1 --2- 172.21.15.79:0/528980421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e0148f30 0x7fd8e00b6630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0009870 tx=0x7fd8d002a270 comp rx=0 tx=0).stop 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8e507b640 1 -- 172.21.15.79:0/528980421 shutdown_connections 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8e507b640 1 --2- 172.21.15.79:0/528980421 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e00b7620 0x7fd8e00b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8e507b640 1 --2- 172.21.15.79:0/528980421 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e0148f30 0x7fd8e00b6630 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8e507b640 1 -- 172.21.15.79:0/528980421 >> 172.21.15.79:0/528980421 conn(0x7fd8e0073b40 msgr2=0x7fd8e0073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:42.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8e507b640 1 -- 172.21.15.79:0/528980421 shutdown_connections 2024-04-26T19:50:42.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.825+0000 7fd8e507b640 1 -- 172.21.15.79:0/528980421 wait complete. 2024-04-26T19:50:42.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.826+0000 7fd8e507b640 1 Processor -- start 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8e507b640 1 -- start start 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8e507b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e006c350 0x7fd8e013af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8e507b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e00b7620 0x7fd8e0162130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8e507b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0148f30 0x7fd8e015dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8e507b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e00b7a60 con 0x7fd8e00b7620 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8e507b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8e00b9440 con 0x7fd8e0148f30 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8e507b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd8e00b7be0 con 0x7fd8e006c350 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8d7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e00b7620 0x7fd8e0162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8d7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e00b7620 0x7fd8e0162130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57996/0 (socket says 172.21.15.79:57996) 2024-04-26T19:50:42.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8d7fff640 1 -- 172.21.15.79:0/3530723871 learned_addr learned my addr 172.21.15.79:0/3530723871 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:42.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8ded9d640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e006c350 0x7fd8e013af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.827+0000 7fd8df59e640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0148f30 0x7fd8e015dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8d7fff640 1 -- 172.21.15.79:0/3530723871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e006c350 msgr2=0x7fd8e013af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8d7fff640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e006c350 0x7fd8e013af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8d7fff640 1 -- 172.21.15.79:0/3530723871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0148f30 msgr2=0x7fd8e015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:42.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8d7fff640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0148f30 0x7fd8e015dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:42.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8d7fff640 1 -- 172.21.15.79:0/3530723871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8e0160700 con 0x7fd8e00b7620 2024-04-26T19:50:42.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8d7fff640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e00b7620 0x7fd8e0162130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800d800 tx=0x7fd8c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:42.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8d77fe640 1 -- 172.21.15.79:0/3530723871 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800b6c0 con 0x7fd8e00b7620 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8d77fe640 1 -- 172.21.15.79:0/3530723871 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8c8019070 con 0x7fd8e00b7620 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8e01609e0 con 0x7fd8e00b7620 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8d77fe640 1 -- 172.21.15.79:0/3530723871 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c800bac0 con 0x7fd8e00b7620 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.828+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8e0161930 con 0x7fd8e00b7620 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.829+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8e00506c0 con 0x7fd8e00b7620 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.829+0000 7fd8d77fe640 1 -- 172.21.15.79:0/3530723871 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8c800bc60 con 0x7fd8e00b7620 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.830+0000 7fd8d77fe640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd8c006c8f0 0x7fd8c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.830+0000 7fd8d77fe640 1 -- 172.21.15.79:0/3530723871 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd8c808c1b0 con 0x7fd8e00b7620 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.830+0000 7fd8ded9d640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd8c006c8f0 0x7fd8c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:42.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.830+0000 7fd8ded9d640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd8c006c8f0 0x7fd8c006edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0034000 tx=0x7fd8d002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:42.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:42.834+0000 7fd8d77fe640 1 -- 172.21.15.79:0/3530723871 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8c8059ce0 con 0x7fd8e00b7620 2024-04-26T19:50:43.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.007+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd8e0161dd0 con 0x7fd8e00b7620 2024-04-26T19:50:43.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.007+0000 7fd8d77fe640 1 -- 172.21.15.79:0/3530723871 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fd8c805dfe0 con 0x7fd8e00b7620 2024-04-26T19:50:43.008 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:50:43.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd8c006c8f0 msgr2=0x7fd8c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd8c006c8f0 0x7fd8c006edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0034000 tx=0x7fd8d002e000 comp rx=0 tx=0).stop 2024-04-26T19:50:43.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e00b7620 msgr2=0x7fd8e0162130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e00b7620 0x7fd8e0162130 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd8c800d800 tx=0x7fd8c800dcc0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 shutdown_connections 2024-04-26T19:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd8c006c8f0 0x7fd8c006edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8e0148f30 0x7fd8e015dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8e00b7620 0x7fd8e0162130 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 --2- 172.21.15.79:0/3530723871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8e006c350 0x7fd8e013af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 >> 172.21.15.79:0/3530723871 conn(0x7fd8e0073b40 msgr2=0x7fd8e0146400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 shutdown_connections 2024-04-26T19:50:43.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.009+0000 7fd8e507b640 1 -- 172.21.15.79:0/3530723871 wait complete. 2024-04-26T19:50:43.021 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:50:43.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.119+0000 7f103dfc6640 1 Processor -- start 2024-04-26T19:50:43.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.119+0000 7f103dfc6640 1 -- start start 2024-04-26T19:50:43.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.120+0000 7f103dfc6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1038142480 0x7f1038163010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.120+0000 7f103dfc6640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1038163e90 con 0x7f1038071f90 2024-04-26T19:50:43.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.120+0000 7f103dfc6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10381635f0 con 0x7f1038160050 2024-04-26T19:50:43.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.120+0000 7f103dfc6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f10381638d0 con 0x7f1038142480 2024-04-26T19:50:43.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.120+0000 7f1036ffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f1038071f90 0x7f1038144b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44356/0 (socket says 172.21.15.79:44356) 2024-04-26T19:50:43.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.120+0000 7f1036ffd640 1 -- 172.21.15.79:0/2521710921 learned_addr learned my addr 172.21.15.79:0/2521710921 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:43.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10377fe640 1 --2- 172.21.15.79:0/2521710921 >> v2:172.21.15.83:3300/0 conn(0x7f1038142480 0x7f1038163010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1892650127 0 0) 0x7f1038163e90 con 0x7f1038071f90 2024-04-26T19:50:43.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1020003520 con 0x7f1038071f90 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4192237239 0 0) 0x7f1020003520 con 0x7f1038071f90 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 >> v2:172.21.15.83:3300/0 conn(0x7f1038142480 msgr2=0x7f1038163010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 --2- 172.21.15.79:0/2521710921 >> v2:172.21.15.83:3300/0 conn(0x7f1038142480 0x7f1038163010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 >> v1:172.21.15.83:6789/0 conn(0x7f1038160050 legacy=0x7f10380d1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1038163bb0 con 0x7f1038071f90 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f102c001670 con 0x7f1038071f90 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 >> v1:172.21.15.79:6789/0 conn(0x7f1038071f90 legacy=0x7f1038144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f10200039b0 0x7f1020003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10200043b0 0x7f1020004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f102000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f10381638d0 con 0x7f1020005080 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f102000bfe0 con 0x7f10200043b0 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f102000c1e0 con 0x7f10200039b0 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f1037fff640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f102000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f10377fe640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f10200039b0 0x7f1020003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f1036ffd640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10200043b0 0x7f1020004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f1037fff640 1 -- 172.21.15.79:0/2521710921 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f10200039b0 msgr2=0x7f1020003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f1037fff640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f10200039b0 0x7f1020003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f1037fff640 1 -- 172.21.15.79:0/2521710921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10200043b0 msgr2=0x7f1020004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f1037fff640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10200043b0 0x7f1020004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.121+0000 7f1037fff640 1 -- 172.21.15.79:0/2521710921 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1020003520 con 0x7f1020005080 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.122+0000 7f1037fff640 1 -- 172.21.15.79:0/2521710921 reap_dead start 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.122+0000 7f1037fff640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f102000b9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1030001f60 tx=0x7f1020005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5ed8dcd265dfbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.122+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1030001b30 con 0x7f1020005080 2024-04-26T19:50:43.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.122+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1030031600 con 0x7f1020005080 2024-04-26T19:50:43.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.122+0000 7f10367fc640 1 -- 172.21.15.79:0/2521710921 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10300319f0 con 0x7f1020005080 2024-04-26T19:50:43.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.123+0000 7f103dfc6640 1 -- 172.21.15.79:0/2521710921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 msgr2=0x7f102000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.123+0000 7f103dfc6640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f102000b9c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1030001f60 tx=0x7f1020005830 comp rx=0 tx=0).stop 2024-04-26T19:50:43.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.123+0000 7f103dfc6640 1 -- 172.21.15.79:0/2521710921 shutdown_connections 2024-04-26T19:50:43.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.123+0000 7f103dfc6640 1 --2- 172.21.15.79:0/2521710921 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f102000b9c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.123+0000 7f103dfc6640 1 -- 172.21.15.79:0/2521710921 >> 172.21.15.79:0/2521710921 conn(0x7f10380b3ed0 msgr2=0x7f10380681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:43.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.123+0000 7f103dfc6640 1 -- 172.21.15.79:0/2521710921 shutdown_connections 2024-04-26T19:50:43.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.123+0000 7f103dfc6640 1 -- 172.21.15.79:0/2521710921 wait complete. 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.124+0000 7f103dfc6640 1 Processor -- start 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.124+0000 7f103dfc6640 1 -- start start 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f103dfc6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f10381557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f103dfc6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1038156b40 0x7f10380aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f103dfc6640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1038072960 0x7f1038072e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f103dfc6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1038163190 con 0x7f1020005080 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f103dfc6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1038163010 con 0x7f1038156b40 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f103dfc6640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1038163310 con 0x7f1038072960 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f10377fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f10381557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f10377fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f10381557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58014/0 (socket says 172.21.15.79:58014) 2024-04-26T19:50:43.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f10377fe640 1 -- 172.21.15.79:0/2934460106 learned_addr learned my addr 172.21.15.79:0/2934460106 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:43.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f10377fe640 1 -- 172.21.15.79:0/2934460106 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1038072960 msgr2=0x7f1038072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f10377fe640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1038072960 0x7f1038072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f10377fe640 1 -- 172.21.15.79:0/2934460106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1038156b40 msgr2=0x7f10380aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f10377fe640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1038156b40 0x7f10380aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f10377fe640 1 -- 172.21.15.79:0/2934460106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1038155d00 con 0x7f1020005080 2024-04-26T19:50:43.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f10377fe640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f10381557c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f10280118e0 tx=0x7f1028011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.125+0000 7f101ffff640 1 -- 172.21.15.79:0/2934460106 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f102800faf0 con 0x7f1020005080 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.126+0000 7f101ffff640 1 -- 172.21.15.79:0/2934460106 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f102800fc90 con 0x7f1020005080 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.126+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10380b0790 con 0x7f1020005080 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.126+0000 7f101ffff640 1 -- 172.21.15.79:0/2934460106 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1028014440 con 0x7f1020005080 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.126+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10380b0d50 con 0x7f1020005080 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.126+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10380506c0 con 0x7f1020005080 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.127+0000 7f101ffff640 1 -- 172.21.15.79:0/2934460106 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1028012040 con 0x7f1020005080 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.127+0000 7f101ffff640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f102406c8f0 0x7f102406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.128+0000 7f1036ffd640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f102406c8f0 0x7f102406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.128+0000 7f101ffff640 1 -- 172.21.15.79:0/2934460106 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f102808c4f0 con 0x7f1020005080 2024-04-26T19:50:43.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.128+0000 7f1036ffd640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f102406c8f0 0x7f102406edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f102c0011e0 tx=0x7f102c001150 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:43.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.132+0000 7f101ffff640 1 -- 172.21.15.79:0/2934460106 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1028019030 con 0x7f1020005080 2024-04-26T19:50:43.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.313+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1038162d80 con 0x7f1020005080 2024-04-26T19:50:43.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.314+0000 7f101ffff640 1 -- 172.21.15.79:0/2934460106 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f102805a020 con 0x7f1020005080 2024-04-26T19:50:43.314 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:50:43.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.316+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f102406c8f0 msgr2=0x7f102406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f102406c8f0 0x7f102406edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f102c0011e0 tx=0x7f102c001150 comp rx=0 tx=0).stop 2024-04-26T19:50:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 msgr2=0x7f10381557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f10381557c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f10280118e0 tx=0x7f1028011da0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 shutdown_connections 2024-04-26T19:50:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f102406c8f0 0x7f102406edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1038072960 0x7f1038072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1038156b40 0x7f10380aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 --2- 172.21.15.79:0/2934460106 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1020005080 0x7f10381557c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 >> 172.21.15.79:0/2934460106 conn(0x7f10380b3ed0 msgr2=0x7f10380cf760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:43.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 shutdown_connections 2024-04-26T19:50:43.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.317+0000 7f103dfc6640 1 -- 172.21.15.79:0/2934460106 wait complete. 2024-04-26T19:50:43.328 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:50:43.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.427+0000 7f4f20140640 1 Processor -- start 2024-04-26T19:50:43.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.427+0000 7f4f20140640 1 -- start start 2024-04-26T19:50:43.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.428+0000 7f4f20140640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f18074490 0x7f4f18160520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.428+0000 7f4f20140640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18145300 0x7f4f18142b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.428+0000 7f4f20140640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f18160a60 con 0x7f4f1806df80 2024-04-26T19:50:43.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.428+0000 7f4f20140640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4f18074870 con 0x7f4f18074490 2024-04-26T19:50:43.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.428+0000 7f4f20140640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1806e500 con 0x7f4f18145300 2024-04-26T19:50:43.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.428+0000 7f4f1e6dd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18145300 0x7f4f18142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.428+0000 7f4f1e6dd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18145300 0x7f4f18142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:54782/0 (socket says 172.21.15.79:54782) 2024-04-26T19:50:43.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.428+0000 7f4f1e6dd640 1 -- 172.21.15.79:0/810357076 learned_addr learned my addr 172.21.15.79:0/810357076 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:43.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.428+0000 7f4f1dedc640 1 --2- 172.21.15.79:0/810357076 >> v2:172.21.15.83:3300/0 conn(0x7f4f18074490 0x7f4f18160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.429+0000 7f4f1ceda640 1 -- 172.21.15.79:0/810357076 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1293123544 0 0) 0x7f4f08001110 con 0x7f4f1806df80 2024-04-26T19:50:43.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.429+0000 7f4f1e6dd640 1 -- 172.21.15.79:0/810357076 >> v2:172.21.15.83:3300/0 conn(0x7f4f18074490 msgr2=0x7f4f18160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:43.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.429+0000 7f4f1e6dd640 1 --2- 172.21.15.79:0/810357076 >> v2:172.21.15.83:3300/0 conn(0x7f4f18074490 0x7f4f18160520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.429+0000 7f4f1e6dd640 1 -- 172.21.15.79:0/810357076 >> v1:172.21.15.83:6789/0 conn(0x7f4f1806df80 legacy=0x7f4f18144bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.429+0000 7f4f1e6dd640 1 -- 172.21.15.79:0/810357076 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1806e7e0 con 0x7f4f18145300 2024-04-26T19:50:43.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.429+0000 7f4f1e6dd640 1 --2- 172.21.15.79:0/810357076 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18145300 0x7f4f18142b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009e50 tx=0x7f4f1002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e7149b8a57b167c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:43.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.429+0000 7f4f1ceda640 1 -- 172.21.15.79:0/810357076 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10036070 con 0x7f4f18145300 2024-04-26T19:50:43.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f1ceda640 1 -- 172.21.15.79:0/810357076 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f10031600 con 0x7f4f18145300 2024-04-26T19:50:43.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f1ceda640 1 -- 172.21.15.79:0/810357076 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f100319d0 con 0x7f4f18145300 2024-04-26T19:50:43.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f20140640 1 -- 172.21.15.79:0/810357076 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18145300 msgr2=0x7f4f18142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f20140640 1 --2- 172.21.15.79:0/810357076 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18145300 0x7f4f18142b90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009e50 tx=0x7f4f1002a710 comp rx=0 tx=0).stop 2024-04-26T19:50:43.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f20140640 1 -- 172.21.15.79:0/810357076 shutdown_connections 2024-04-26T19:50:43.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f20140640 1 --2- 172.21.15.79:0/810357076 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18145300 0x7f4f18142b90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f20140640 1 --2- 172.21.15.79:0/810357076 >> v2:172.21.15.83:3300/0 conn(0x7f4f18074490 0x7f4f18160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f20140640 1 -- 172.21.15.79:0/810357076 >> 172.21.15.79:0/810357076 conn(0x7f4f180685d0 msgr2=0x7f4f180d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:43.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f20140640 1 -- 172.21.15.79:0/810357076 shutdown_connections 2024-04-26T19:50:43.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.430+0000 7f4f20140640 1 -- 172.21.15.79:0/810357076 wait complete. 2024-04-26T19:50:43.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.431+0000 7f4f20140640 1 Processor -- start 2024-04-26T19:50:43.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.431+0000 7f4f20140640 1 -- start start 2024-04-26T19:50:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f20140640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1806df80 0x7f4f180aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f20140640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18074490 0x7f4f180b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f20140640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f18145300 0x7f4f18158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f20140640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1806f080 con 0x7f4f18145300 2024-04-26T19:50:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f20140640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1806eda0 con 0x7f4f1806df80 2024-04-26T19:50:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f20140640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1806f5a0 con 0x7f4f18074490 2024-04-26T19:50:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f1dedc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1806df80 0x7f4f180aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f1e6dd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f18145300 0x7f4f18158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f1dedc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1806df80 0x7f4f180aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:34058/0 (socket says 172.21.15.79:34058) 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f1dedc640 1 -- 172.21.15.79:0/3414646223 learned_addr learned my addr 172.21.15.79:0/3414646223 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f1d6db640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18074490 0x7f4f180b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f1e6dd640 1 -- 172.21.15.79:0/3414646223 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18074490 msgr2=0x7f4f180b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.432+0000 7f4f1e6dd640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18074490 0x7f4f180b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f1e6dd640 1 -- 172.21.15.79:0/3414646223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1806df80 msgr2=0x7f4f180aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f1e6dd640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1806df80 0x7f4f180aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f1e6dd640 1 -- 172.21.15.79:0/3414646223 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f18159510 con 0x7f4f18145300 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f1dedc640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1806df80 0x7f4f180aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f1d6db640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18074490 0x7f4f180b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f1e6dd640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f18145300 0x7f4f18158440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f10030000 tx=0x7f4f10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f0e7fc640 1 -- 172.21.15.79:0/3414646223 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10041070 con 0x7f4f18145300 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f0e7fc640 1 -- 172.21.15.79:0/3414646223 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f10003990 con 0x7f4f18145300 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f0e7fc640 1 -- 172.21.15.79:0/3414646223 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10036040 con 0x7f4f18145300 2024-04-26T19:50:43.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f181597f0 con 0x7f4f18145300 2024-04-26T19:50:43.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.433+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f18131e10 con 0x7f4f18145300 2024-04-26T19:50:43.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.434+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f180506c0 con 0x7f4f18145300 2024-04-26T19:50:43.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.434+0000 7f4f0e7fc640 1 -- 172.21.15.79:0/3414646223 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f10007820 con 0x7f4f18145300 2024-04-26T19:50:43.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.435+0000 7f4f0e7fc640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.435+0000 7f4f0e7fc640 1 -- 172.21.15.79:0/3414646223 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4f100ac740 con 0x7f4f18145300 2024-04-26T19:50:43.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.435+0000 7f4f1dedc640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.435+0000 7f4f1dedc640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4f140112c0 tx=0x7f4f14009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:43.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.439+0000 7f4f0e7fc640 1 -- 172.21.15.79:0/3414646223 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f1004b020 con 0x7f4f18145300 2024-04-26T19:50:43.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.615+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f18071810 con 0x7f4f18145300 2024-04-26T19:50:43.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.615+0000 7f4f0e7fc640 1 -- 172.21.15.79:0/3414646223 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f4f1007a1e0 con 0x7f4f18145300 2024-04-26T19:50:43.617 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:50:43.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 msgr2=0x7f4efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4f140112c0 tx=0x7f4f14009190 comp rx=0 tx=0).stop 2024-04-26T19:50:43.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f18145300 msgr2=0x7f4f18158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f18145300 0x7f4f18158440 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4f10030000 tx=0x7f4f10004a40 comp rx=0 tx=0).stop 2024-04-26T19:50:43.618 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 shutdown_connections 2024-04-26T19:50:43.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f18145300 0x7f4f18158440 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f18074490 0x7f4f180b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 --2- 172.21.15.79:0/3414646223 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f1806df80 0x7f4f180aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 >> 172.21.15.79:0/3414646223 conn(0x7f4f180685d0 msgr2=0x7f4f181621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:43.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.617+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 shutdown_connections 2024-04-26T19:50:43.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.618+0000 7f4f20140640 1 -- 172.21.15.79:0/3414646223 wait complete. 2024-04-26T19:50:43.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:50:43.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.727+0000 7f3684787640 1 Processor -- start 2024-04-26T19:50:43.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.727+0000 7f3684787640 1 -- start start 2024-04-26T19:50:43.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.728+0000 7f3684787640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f367c15c1e0 0x7f367c073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.728+0000 7f3684787640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f367c06f920 0x7f367c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.728+0000 7f3684787640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f367c072620 con 0x7f367c0718f0 2024-04-26T19:50:43.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.728+0000 7f3684787640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f367c0730d0 con 0x7f367c06f920 2024-04-26T19:50:43.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.728+0000 7f3684787640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f367c0733b0 con 0x7f367c15c1e0 2024-04-26T19:50:43.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3681d22640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f367c06f920 0x7f367c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3681d22640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f367c06f920 0x7f367c06fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58038/0 (socket says 172.21.15.79:58038) 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3681d22640 1 -- 172.21.15.79:0/1739932423 learned_addr learned my addr 172.21.15.79:0/1739932423 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3682d24640 1 --1- 172.21.15.79:0/1739932423 >> v1:172.21.15.79:6789/0 conn(0x7f367c0718f0 0x7f367c071d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44372/0 (socket says 172.21.15.79:44372) 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3682523640 1 --2- 172.21.15.79:0/1739932423 >> v2:172.21.15.83:3300/0 conn(0x7f367c15c1e0 0x7f367c073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3681d22640 1 -- 172.21.15.79:0/1739932423 >> v2:172.21.15.83:3300/0 conn(0x7f367c15c1e0 msgr2=0x7f367c073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3681d22640 1 --2- 172.21.15.79:0/1739932423 >> v2:172.21.15.83:3300/0 conn(0x7f367c15c1e0 0x7f367c073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3681d22640 1 -- 172.21.15.79:0/1739932423 >> v1:172.21.15.79:6789/0 conn(0x7f367c0718f0 legacy=0x7f367c071d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3681521640 1 -- 172.21.15.79:0/1739932423 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3766131990 0 0) 0x7f367c072620 con 0x7f367c0718f0 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.729+0000 7f3681d22640 1 -- 172.21.15.79:0/1739932423 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c073690 con 0x7f367c06f920 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3681d22640 1 --2- 172.21.15.79:0/1739932423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 0x7f367c06fdb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f367c0d1160 tx=0x7f366c02c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bf4045ef2c63ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3681521640 1 -- 172.21.15.79:0/1739932423 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c032710 con 0x7f367c06f920 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3681521640 1 -- 172.21.15.79:0/1739932423 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f366c0328b0 con 0x7f367c06f920 2024-04-26T19:50:43.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3681521640 1 -- 172.21.15.79:0/1739932423 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366c032ca0 con 0x7f367c06f920 2024-04-26T19:50:43.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3684787640 1 -- 172.21.15.79:0/1739932423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 msgr2=0x7f367c06fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3684787640 1 --2- 172.21.15.79:0/1739932423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 0x7f367c06fdb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f367c0d1160 tx=0x7f366c02c810 comp rx=0 tx=0).stop 2024-04-26T19:50:43.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3684787640 1 -- 172.21.15.79:0/1739932423 shutdown_connections 2024-04-26T19:50:43.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3684787640 1 --2- 172.21.15.79:0/1739932423 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 0x7f367c06fdb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3684787640 1 --2- 172.21.15.79:0/1739932423 >> v2:172.21.15.83:3300/0 conn(0x7f367c15c1e0 0x7f367c073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3684787640 1 -- 172.21.15.79:0/1739932423 >> 172.21.15.79:0/1739932423 conn(0x7f367c0b2540 msgr2=0x7f367c0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:43.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3684787640 1 -- 172.21.15.79:0/1739932423 shutdown_connections 2024-04-26T19:50:43.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.730+0000 7f3684787640 1 -- 172.21.15.79:0/1739932423 wait complete. 2024-04-26T19:50:43.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.731+0000 7f3684787640 1 Processor -- start 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.731+0000 7f3684787640 1 -- start start 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3684787640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 0x7f367c1f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3684787640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f367c0718f0 0x7f367c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3684787640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f367c15c1e0 0x7f367c14b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3684787640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f367c0ce020 con 0x7f367c06f920 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3684787640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f367c0cdea0 con 0x7f367c0718f0 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3684787640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f367c0ce1a0 con 0x7f367c15c1e0 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3682523640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 0x7f367c1f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3682523640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 0x7f367c1f5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58050/0 (socket says 172.21.15.79:58050) 2024-04-26T19:50:43.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3682523640 1 -- 172.21.15.79:0/3623385023 learned_addr learned my addr 172.21.15.79:0/3623385023 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3682d24640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f367c15c1e0 0x7f367c14b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.732+0000 7f3681d22640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f367c0718f0 0x7f367c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3682523640 1 -- 172.21.15.79:0/3623385023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f367c15c1e0 msgr2=0x7f367c14b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3682523640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f367c15c1e0 0x7f367c14b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3682523640 1 -- 172.21.15.79:0/3623385023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f367c0718f0 msgr2=0x7f367c1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3682523640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f367c0718f0 0x7f367c1599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3682523640 1 -- 172.21.15.79:0/3623385023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367c1500a0 con 0x7f367c06f920 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3682523640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 0x7f367c1f5400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3674007e60 tx=0x7f367400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3672ffd640 1 -- 172.21.15.79:0/3623385023 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674014620 con 0x7f367c06f920 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3672ffd640 1 -- 172.21.15.79:0/3623385023 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3674019070 con 0x7f367c06f920 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367c159f20 con 0x7f367c06f920 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3672ffd640 1 -- 172.21.15.79:0/3623385023 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674014a20 con 0x7f367c06f920 2024-04-26T19:50:43.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.733+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f367c15a1d0 con 0x7f367c06f920 2024-04-26T19:50:43.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.734+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f367c0506c0 con 0x7f367c06f920 2024-04-26T19:50:43.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.734+0000 7f3672ffd640 1 -- 172.21.15.79:0/3623385023 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3674014bc0 con 0x7f367c06f920 2024-04-26T19:50:43.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.735+0000 7f3672ffd640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f366006c8f0 0x7f366006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:43.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.735+0000 7f3672ffd640 1 -- 172.21.15.79:0/3623385023 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f367408c5b0 con 0x7f367c06f920 2024-04-26T19:50:43.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.735+0000 7f3681d22640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f366006c8f0 0x7f366006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:43.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.735+0000 7f3681d22640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f366006c8f0 0x7f366006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f366c037000 tx=0x7f366c032000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:43.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.739+0000 7f3672ffd640 1 -- 172.21.15.79:0/3623385023 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f367405a0e0 con 0x7f367c06f920 2024-04-26T19:50:43.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.912+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f367c0cda70 con 0x7f367c06f920 2024-04-26T19:50:43.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.912+0000 7f3672ffd640 1 -- 172.21.15.79:0/3623385023 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f367405e3e0 con 0x7f367c06f920 2024-04-26T19:50:43.913 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:50:43.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f366006c8f0 msgr2=0x7f366006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f366006c8f0 0x7f366006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f366c037000 tx=0x7f366c032000 comp rx=0 tx=0).stop 2024-04-26T19:50:43.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 msgr2=0x7f367c1f5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:43.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 0x7f367c1f5400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3674007e60 tx=0x7f367400e4b0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 shutdown_connections 2024-04-26T19:50:43.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f366006c8f0 0x7f366006edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f367c15c1e0 0x7f367c14b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f367c0718f0 0x7f367c1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 --2- 172.21.15.79:0/3623385023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f367c06f920 0x7f367c1f5400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:43.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 >> 172.21.15.79:0/3623385023 conn(0x7f367c0b2540 msgr2=0x7f367c0d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:43.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 shutdown_connections 2024-04-26T19:50:43.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:43.914+0000 7f3684787640 1 -- 172.21.15.79:0/3623385023 wait complete. 2024-04-26T19:50:43.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T19:50:44.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.022+0000 7fe43cdf7640 1 Processor -- start 2024-04-26T19:50:44.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.022+0000 7fe43cdf7640 1 -- start start 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe43cdf7640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe438145e20 0x7fe4380d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe43cdf7640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe4380b64f0 0x7fe4380b6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe43cdf7640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4380d06a0 con 0x7fe4380c79c0 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe43cdf7640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fe4380d0460 con 0x7fe4380b64f0 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe43cdf7640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe4380d0980 con 0x7fe438145e20 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe436d9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe4380b64f0 0x7fe4380b6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe435d9b640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe4380c79c0 0x7fe4380cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:44384/0 (socket says 172.21.15.79:44384) 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe435d9b640 1 -- 172.21.15.79:0/164517823 learned_addr learned my addr 172.21.15.79:0/164517823 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe43659c640 1 --2- 172.21.15.79:0/164517823 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe438145e20 0x7fe4380d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:44.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.023+0000 7fe436d9d640 1 -- 172.21.15.79:0/164517823 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe438145e20 msgr2=0x7fe4380d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.024+0000 7fe436d9d640 1 --2- 172.21.15.79:0/164517823 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe438145e20 0x7fe4380d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.024+0000 7fe436d9d640 1 -- 172.21.15.79:0/164517823 >> v1:172.21.15.79:6789/0 conn(0x7fe4380c79c0 legacy=0x7fe4380cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.024+0000 7fe436d9d640 1 -- 172.21.15.79:0/164517823 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4380d0c60 con 0x7fe4380b64f0 2024-04-26T19:50:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.024+0000 7fe43559a640 1 -- 172.21.15.79:0/164517823 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2429595638 0 0) 0x7fe4380d06a0 con 0x7fe4380c79c0 2024-04-26T19:50:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.024+0000 7fe43659c640 1 --2- 172.21.15.79:0/164517823 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe438145e20 0x7fe4380d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.024+0000 7fe436d9d640 1 --2- 172.21.15.79:0/164517823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380b64f0 0x7fe4380b6980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe4300097a0 tx=0x7fe43002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d99b87b77efa55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.024+0000 7fe43559a640 1 -- 172.21.15.79:0/164517823 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430037070 con 0x7fe4380b64f0 2024-04-26T19:50:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.024+0000 7fe43559a640 1 -- 172.21.15.79:0/164517823 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe43002ae50 con 0x7fe4380b64f0 2024-04-26T19:50:44.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.024+0000 7fe43559a640 1 -- 172.21.15.79:0/164517823 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430032660 con 0x7fe4380b64f0 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.025+0000 7fe43cdf7640 1 -- 172.21.15.79:0/164517823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380b64f0 msgr2=0x7fe4380b6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.025+0000 7fe43cdf7640 1 --2- 172.21.15.79:0/164517823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380b64f0 0x7fe4380b6980 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe4300097a0 tx=0x7fe43002a100 comp rx=0 tx=0).stop 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.025+0000 7fe43cdf7640 1 -- 172.21.15.79:0/164517823 shutdown_connections 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.025+0000 7fe43cdf7640 1 --2- 172.21.15.79:0/164517823 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380b64f0 0x7fe4380b6980 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.025+0000 7fe43cdf7640 1 --2- 172.21.15.79:0/164517823 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe438145e20 0x7fe4380d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.025+0000 7fe43cdf7640 1 -- 172.21.15.79:0/164517823 >> 172.21.15.79:0/164517823 conn(0x7fe43806c770 msgr2=0x7fe43806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.025+0000 7fe43cdf7640 1 -- 172.21.15.79:0/164517823 shutdown_connections 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.025+0000 7fe43cdf7640 1 -- 172.21.15.79:0/164517823 wait complete. 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.026+0000 7fe43cdf7640 1 Processor -- start 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.026+0000 7fe43cdf7640 1 -- start start 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe43cdf7640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4380b64f0 0x7fe4380c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe43cdf7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380c79c0 0x7fe438131de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe43cdf7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe438145e20 0x7fe43815fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:44.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe43cdf7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe43815d660 con 0x7fe4380c79c0 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe43cdf7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4380d1220 con 0x7fe438145e20 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe43cdf7640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe43815d7e0 con 0x7fe4380b64f0 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe435d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380c79c0 0x7fe438131de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe435d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380c79c0 0x7fe438131de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:58076/0 (socket says 172.21.15.79:58076) 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe435d9b640 1 -- 172.21.15.79:0/3008491556 learned_addr learned my addr 172.21.15.79:0/3008491556 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe436d9d640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe438145e20 0x7fe43815fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe43659c640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4380b64f0 0x7fe4380c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.027+0000 7fe435d9b640 1 -- 172.21.15.79:0/3008491556 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4380b64f0 msgr2=0x7fe4380c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe435d9b640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4380b64f0 0x7fe4380c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe435d9b640 1 -- 172.21.15.79:0/3008491556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe438145e20 msgr2=0x7fe43815fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe435d9b640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe438145e20 0x7fe43815fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe435d9b640 1 -- 172.21.15.79:0/3008491556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4380ce6b0 con 0x7fe4380c79c0 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe435d9b640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380c79c0 0x7fe438131de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00c4d0 tx=0x7fe42c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe426ffd640 1 -- 172.21.15.79:0/3008491556 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c018070 con 0x7fe4380c79c0 2024-04-26T19:50:44.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe426ffd640 1 -- 172.21.15.79:0/3008491556 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe42c007500 con 0x7fe4380c79c0 2024-04-26T19:50:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe426ffd640 1 -- 172.21.15.79:0/3008491556 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe42c004a20 con 0x7fe4380c79c0 2024-04-26T19:50:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe438132320 con 0x7fe4380c79c0 2024-04-26T19:50:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.028+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4380cec30 con 0x7fe4380c79c0 2024-04-26T19:50:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.029+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4380506c0 con 0x7fe4380c79c0 2024-04-26T19:50:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.029+0000 7fe426ffd640 1 -- 172.21.15.79:0/3008491556 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe42c01b070 con 0x7fe4380c79c0 2024-04-26T19:50:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.029+0000 7fe426ffd640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.030+0000 7fe426ffd640 1 -- 172.21.15.79:0/3008491556 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe42c08b0e0 con 0x7fe4380c79c0 2024-04-26T19:50:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.030+0000 7fe43659c640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.030+0000 7fe43659c640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe428009fd0 tx=0x7fe428009870 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:44.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.034+0000 7fe426ffd640 1 -- 172.21.15.79:0/3008491556 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe42c058b80 con 0x7fe4380c79c0 2024-04-26T19:50:44.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.224+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe438073650 con 0x7fe4380c79c0 2024-04-26T19:50:44.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.224+0000 7fe426ffd640 1 -- 172.21.15.79:0/3008491556 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fe42c05ce80 con 0x7fe4380c79c0 2024-04-26T19:50:44.225 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2024-04-26T19:50:44.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe41806c8f0 msgr2=0x7fe41806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:44.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe41806c8f0 0x7fe41806edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe428009fd0 tx=0x7fe428009870 comp rx=0 tx=0).stop 2024-04-26T19:50:44.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380c79c0 msgr2=0x7fe438131de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:44.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380c79c0 0x7fe438131de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe42c00c4d0 tx=0x7fe42c00c990 comp rx=0 tx=0).stop 2024-04-26T19:50:44.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 shutdown_connections 2024-04-26T19:50:44.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe41806c8f0 0x7fe41806edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:44.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe438145e20 0x7fe43815fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:44.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4380c79c0 0x7fe438131de0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:44.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 --2- 172.21.15.79:0/3008491556 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4380b64f0 0x7fe4380c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:44.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 >> 172.21.15.79:0/3008491556 conn(0x7fe43806c770 msgr2=0x7fe43815cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:44.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 shutdown_connections 2024-04-26T19:50:44.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:44.226+0000 7fe43cdf7640 1 -- 172.21.15.79:0/3008491556 wait complete. 2024-04-26T19:50:44.238 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi079.front.sepia.ceph.com... 2024-04-26T19:50:44.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:44.238 DEBUG:teuthology.orchestra.run.smithi079:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-26T19:50:44.260 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi079.stderr:ceph-fuse[42193]: fuse finished with error 0 and tester_r 0 2024-04-26T19:50:44.261 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:50:50.363 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:50:50.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:50:50.364 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-26T19:50:50.390 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:50.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.538+0000 7fe4b53f2640 1 Processor -- start 2024-04-26T19:50:50.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.539+0000 7fe4b53f2640 1 -- start start 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.539+0000 7fe4b53f2640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 0x7fe4b00b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.539+0000 7fe4b53f2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4b00735a0 0x7fe4b00b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.539+0000 7fe4b53f2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b00ce1a0 con 0x7fe4b0156670 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.539+0000 7fe4b53f2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe4b0073110 con 0x7fe4b00735a0 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.539+0000 7fe4b53f2640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b00cdec0 con 0x7fe4b00cd9c0 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.540+0000 7fe4af7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4b00735a0 0x7fe4b00b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.540+0000 7fe4ae7fc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 0x7fe4b00b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.540+0000 7fe4aeffd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe4b0156670 0x7fe4b015d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:50974/0 (socket says 172.21.15.79:50974) 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.540+0000 7fe4aeffd640 1 -- 172.21.15.79:0/3261565040 learned_addr learned my addr 172.21.15.79:0/3261565040 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.540+0000 7fe4ae7fc640 1 -- 172.21.15.79:0/3261565040 >> v2:172.21.15.83:3300/0 conn(0x7fe4b00735a0 msgr2=0x7fe4b00b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.540+0000 7fe4ae7fc640 1 --2- 172.21.15.79:0/3261565040 >> v2:172.21.15.83:3300/0 conn(0x7fe4b00735a0 0x7fe4b00b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.540+0000 7fe4ae7fc640 1 -- 172.21.15.79:0/3261565040 >> v1:172.21.15.83:6789/0 conn(0x7fe4b0156670 legacy=0x7fe4b015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.540+0000 7fe4ae7fc640 1 -- 172.21.15.79:0/3261565040 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b0072ac0 con 0x7fe4b00cd9c0 2024-04-26T19:50:50.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.540+0000 7fe4af7fe640 1 --2- 172.21.15.79:0/3261565040 >> v2:172.21.15.83:3300/0 conn(0x7fe4b00735a0 0x7fe4b00b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:50.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.541+0000 7fe4ae7fc640 1 --2- 172.21.15.79:0/3261565040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 0x7fe4b00b3de0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40097a0 tx=0x7fe4a402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec646b2bb7e201b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:50.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.541+0000 7fe4adffb640 1 -- 172.21.15.79:0/3261565040 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4037070 con 0x7fe4b00cd9c0 2024-04-26T19:50:50.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.541+0000 7fe4adffb640 1 -- 172.21.15.79:0/3261565040 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a4032440 con 0x7fe4b00cd9c0 2024-04-26T19:50:50.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.541+0000 7fe4adffb640 1 -- 172.21.15.79:0/3261565040 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4032810 con 0x7fe4b00cd9c0 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.541+0000 7fe4b53f2640 1 -- 172.21.15.79:0/3261565040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 msgr2=0x7fe4b00b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.541+0000 7fe4b53f2640 1 --2- 172.21.15.79:0/3261565040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 0x7fe4b00b3de0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40097a0 tx=0x7fe4a402a180 comp rx=0 tx=0).stop 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.542+0000 7fe4b53f2640 1 -- 172.21.15.79:0/3261565040 shutdown_connections 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.542+0000 7fe4b53f2640 1 --2- 172.21.15.79:0/3261565040 >> v2:172.21.15.83:3300/0 conn(0x7fe4b00735a0 0x7fe4b00b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.542+0000 7fe4b53f2640 1 --2- 172.21.15.79:0/3261565040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 0x7fe4b00b3de0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.542+0000 7fe4b53f2640 1 -- 172.21.15.79:0/3261565040 >> 172.21.15.79:0/3261565040 conn(0x7fe4b00b39d0 msgr2=0x7fe4b006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.542+0000 7fe4b53f2640 1 -- 172.21.15.79:0/3261565040 shutdown_connections 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.542+0000 7fe4b53f2640 1 -- 172.21.15.79:0/3261565040 wait complete. 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.543+0000 7fe4b53f2640 1 Processor -- start 2024-04-26T19:50:50.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.543+0000 7fe4b53f2640 1 -- start start 2024-04-26T19:50:50.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4b53f2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b00735a0 0x7fe4b01530f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4b53f2640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 0x7fe4b01542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4b53f2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4b0156670 0x7fe4b015b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4b53f2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b0156bd0 con 0x7fe4b00735a0 2024-04-26T19:50:50.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4b53f2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b0156a50 con 0x7fe4b0156670 2024-04-26T19:50:50.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4b53f2640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b00cf650 con 0x7fe4b00cd9c0 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b00735a0 0x7fe4b01530f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b00735a0 0x7fe4b01530f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38584/0 (socket says 172.21.15.79:38584) 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4aeffd640 1 -- 172.21.15.79:0/1379630407 learned_addr learned my addr 172.21.15.79:0/1379630407 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4af7fe640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4b0156670 0x7fe4b015b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4ae7fc640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 0x7fe4b01542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4aeffd640 1 -- 172.21.15.79:0/1379630407 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 msgr2=0x7fe4b01542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4aeffd640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 0x7fe4b01542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4aeffd640 1 -- 172.21.15.79:0/1379630407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4b0156670 msgr2=0x7fe4b015b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4aeffd640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4b0156670 0x7fe4b015b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4aeffd640 1 -- 172.21.15.79:0/1379630407 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b014bab0 con 0x7fe4b00735a0 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4aeffd640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b00735a0 0x7fe4b01530f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000ca70 tx=0x7fe4a000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4977fe640 1 -- 172.21.15.79:0/1379630407 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0017070 con 0x7fe4b00735a0 2024-04-26T19:50:50.547 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.544+0000 7fe4977fe640 1 -- 172.21.15.79:0/1379630407 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a000abd0 con 0x7fe4b00735a0 2024-04-26T19:50:50.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.545+0000 7fe4977fe640 1 -- 172.21.15.79:0/1379630407 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0012400 con 0x7fe4b00735a0 2024-04-26T19:50:50.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.545+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b014bd90 con 0x7fe4b00735a0 2024-04-26T19:50:50.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.545+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b014c350 con 0x7fe4b00735a0 2024-04-26T19:50:50.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.546+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b00506c0 con 0x7fe4b00735a0 2024-04-26T19:50:50.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.546+0000 7fe4977fe640 1 -- 172.21.15.79:0/1379630407 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4a001a070 con 0x7fe4b00735a0 2024-04-26T19:50:50.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.546+0000 7fe4977fe640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe49c06c8f0 0x7fe49c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.546+0000 7fe4977fe640 1 -- 172.21.15.79:0/1379630407 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe4a008bb10 con 0x7fe4b00735a0 2024-04-26T19:50:50.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.546+0000 7fe4ae7fc640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe49c06c8f0 0x7fe49c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.547+0000 7fe4ae7fc640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe49c06c8f0 0x7fe49c06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4034000 tx=0x7fe4a402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:50.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.551+0000 7fe4977fe640 1 -- 172.21.15.79:0/1379630407 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4a0059c90 con 0x7fe4b00735a0 2024-04-26T19:50:50.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.725+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4b0070380 con 0x7fe4b00735a0 2024-04-26T19:50:50.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.726+0000 7fe4977fe640 1 -- 172.21.15.79:0/1379630407 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fe4a0059590 con 0x7fe4b00735a0 2024-04-26T19:50:50.727 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:50.727 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:50.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.728+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe49c06c8f0 msgr2=0x7fe49c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.728+0000 7fe4b53f2640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe49c06c8f0 0x7fe49c06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4034000 tx=0x7fe4a402e000 comp rx=0 tx=0).stop 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.728+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b00735a0 msgr2=0x7fe4b01530f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.728+0000 7fe4b53f2640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b00735a0 0x7fe4b01530f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000ca70 tx=0x7fe4a000cf30 comp rx=0 tx=0).stop 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.729+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 shutdown_connections 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.729+0000 7fe4b53f2640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe49c06c8f0 0x7fe49c06edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.729+0000 7fe4b53f2640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4b0156670 0x7fe4b015b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.729+0000 7fe4b53f2640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4b00cd9c0 0x7fe4b01542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.729+0000 7fe4b53f2640 1 --2- 172.21.15.79:0/1379630407 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4b00735a0 0x7fe4b01530f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.729+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 >> 172.21.15.79:0/1379630407 conn(0x7fe4b00b39d0 msgr2=0x7fe4b01600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:50.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.729+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 shutdown_connections 2024-04-26T19:50:50.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.729+0000 7fe4b53f2640 1 -- 172.21.15.79:0/1379630407 wait complete. 2024-04-26T19:50:50.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:50.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.836+0000 7fb9015af640 1 Processor -- start 2024-04-26T19:50:50.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.836+0000 7fb9015af640 1 -- start start 2024-04-26T19:50:50.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb9015af640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc06ff00 0x7fb8fc148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb9015af640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8fc0b1290 con 0x7fb8fc0d2cd0 2024-04-26T19:50:50.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb9015af640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8fc0d4090 con 0x7fb8fc06fb20 2024-04-26T19:50:50.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb9015af640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb8fc0d3110 con 0x7fb8fc06ff00 2024-04-26T19:50:50.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8fb7fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fb8fc0d2cd0 0x7fb8fc0aebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46190/0 (socket says 172.21.15.79:46190) 2024-04-26T19:50:50.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8fb7fe640 1 -- 172.21.15.79:0/4211636469 learned_addr learned my addr 172.21.15.79:0/4211636469 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:50.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8fa7fc640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc06ff00 0x7fb8fc148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 944011574 0 0) 0x7fb8fc0b1290 con 0x7fb8fc0d2cd0 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8e4003520 con 0x7fb8fc0d2cd0 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4122827615 0 0) 0x7fb8e4003520 con 0x7fb8fc0d2cd0 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc06ff00 msgr2=0x7fb8fc148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc06ff00 0x7fb8fc148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 >> v1:172.21.15.83:6789/0 conn(0x7fb8fc06fb20 legacy=0x7fb8fc1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc0d4370 con 0x7fb8fc0d2cd0 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fb8f40019f0 con 0x7fb8fc0d2cd0 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 >> v1:172.21.15.79:6789/0 conn(0x7fb8fc0d2cd0 legacy=0x7fb8fc0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc06ff00 0x7fb8e40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4003f60 0x7fb8e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8e4004d40 0x7fb8e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc0d3110 con 0x7fb8e4003f60 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e400bb30 con 0x7fb8e4004d40 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e400bd30 con 0x7fb8fc06ff00 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8fa7fc640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4003f60 0x7fb8e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8faffd640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc06ff00 0x7fb8e40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.837+0000 7fb8fb7fe640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8e4004d40 0x7fb8e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.838+0000 7fb8fa7fc640 1 -- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc06ff00 msgr2=0x7fb8e40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.838+0000 7fb8fa7fc640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc06ff00 0x7fb8e40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.838+0000 7fb8fa7fc640 1 -- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8e4004d40 msgr2=0x7fb8e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.838+0000 7fb8fa7fc640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8e4004d40 0x7fb8e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.838+0000 7fb8fa7fc640 1 -- 172.21.15.79:0/4211636469 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e4003520 con 0x7fb8e4003f60 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.838+0000 7fb8fa7fc640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4003f60 0x7fb8e4004800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009ac0 tx=0x7fb8e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9291ee508b2476a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.838+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0036070 con 0x7fb8e4003f60 2024-04-26T19:50:50.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.838+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f0031400 con 0x7fb8e4003f60 2024-04-26T19:50:50.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.838+0000 7fb8f9ffb640 1 -- 172.21.15.79:0/4211636469 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f0009690 con 0x7fb8e4003f60 2024-04-26T19:50:50.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.839+0000 7fb9015af640 1 -- 172.21.15.79:0/4211636469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4003f60 msgr2=0x7fb8e4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.839+0000 7fb9015af640 1 --2- 172.21.15.79:0/4211636469 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8e4003f60 0x7fb8e4004800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009ac0 tx=0x7fb8e4005830 comp rx=0 tx=0).stop 2024-04-26T19:50:50.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.839+0000 7fb8fb7fe640 1 -- 172.21.15.79:0/4211636469 reap_dead start 2024-04-26T19:50:50.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.839+0000 7fb9015af640 1 -- 172.21.15.79:0/4211636469 shutdown_connections 2024-04-26T19:50:50.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.839+0000 7fb9015af640 1 -- 172.21.15.79:0/4211636469 >> 172.21.15.79:0/4211636469 conn(0x7fb8fc071c30 msgr2=0x7fb8fc06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:50.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.839+0000 7fb9015af640 1 -- 172.21.15.79:0/4211636469 shutdown_connections 2024-04-26T19:50:50.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.839+0000 7fb9015af640 1 -- 172.21.15.79:0/4211636469 wait complete. 2024-04-26T19:50:50.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.840+0000 7fb9015af640 1 Processor -- start 2024-04-26T19:50:50.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.840+0000 7fb9015af640 1 -- start start 2024-04-26T19:50:50.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb9015af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8e4003f60 0x7fb8fc146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb9015af640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc161b90 0x7fb8fc144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb9015af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8fc15ebf0 0x7fb8fc15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb9015af640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc0d4c10 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb9015af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8fc0d4930 con 0x7fb8e4003f60 2024-04-26T19:50:50.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb9015af640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb8fc148fd0 con 0x7fb8fc161b90 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8fc15ebf0 0x7fb8fc15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8fc15ebf0 0x7fb8fc15f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38600/0 (socket says 172.21.15.79:38600) 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8fb7fe640 1 -- 172.21.15.79:0/3994923265 learned_addr learned my addr 172.21.15.79:0/3994923265 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8faffd640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8e4003f60 0x7fb8fc146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8fa7fc640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc161b90 0x7fb8fc144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8fb7fe640 1 -- 172.21.15.79:0/3994923265 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc161b90 msgr2=0x7fb8fc144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8fb7fe640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc161b90 0x7fb8fc144d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8fb7fe640 1 -- 172.21.15.79:0/3994923265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8e4003f60 msgr2=0x7fb8fc146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8fb7fe640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8e4003f60 0x7fb8fc146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.841+0000 7fb8fb7fe640 1 -- 172.21.15.79:0/3994923265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8fc1f6020 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.842+0000 7fb8fb7fe640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8fc15ebf0 0x7fb8fc15f0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc149540 tx=0x7fb8f4007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.842+0000 7fb8e37fe640 1 -- 172.21.15.79:0/3994923265 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400abe0 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.842+0000 7fb8e37fe640 1 -- 172.21.15.79:0/3994923265 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f4018070 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.842+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8fc1474e0 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.842+0000 7fb8e37fe640 1 -- 172.21.15.79:0/3994923265 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40133f0 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.842+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8fc1f6290 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.843+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8fc0506c0 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.843+0000 7fb8e37fe640 1 -- 172.21.15.79:0/3994923265 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8f4013590 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.844+0000 7fb8e37fe640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8e806c920 0x7fb8e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.844+0000 7fb8faffd640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8e806c920 0x7fb8e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.844+0000 7fb8e37fe640 1 -- 172.21.15.79:0/3994923265 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb8f408c2a0 con 0x7fb8fc15ebf0 2024-04-26T19:50:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.844+0000 7fb8faffd640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8e806c920 0x7fb8e806ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0017e0 tx=0x7fb8ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:50.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:50.848+0000 7fb8e37fe640 1 -- 172.21.15.79:0/3994923265 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8f4059dd0 con 0x7fb8fc15ebf0 2024-04-26T19:50:51.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.022+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8fc0d3700 con 0x7fb8fc15ebf0 2024-04-26T19:50:51.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.023+0000 7fb8e37fe640 1 -- 172.21.15.79:0/3994923265 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fb8f408c0c0 con 0x7fb8fc15ebf0 2024-04-26T19:50:51.023 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:51.024 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8e806c920 msgr2=0x7fb8e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8e806c920 0x7fb8e806ede0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0017e0 tx=0x7fb8ec009040 comp rx=0 tx=0).stop 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8fc15ebf0 msgr2=0x7fb8fc15f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8fc15ebf0 0x7fb8fc15f0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc149540 tx=0x7fb8f4007bd0 comp rx=0 tx=0).stop 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 shutdown_connections 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8e806c920 0x7fb8e806ede0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8fc15ebf0 0x7fb8fc15f0a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8fc161b90 0x7fb8fc144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 --2- 172.21.15.79:0/3994923265 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8e4003f60 0x7fb8fc146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:51.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 >> 172.21.15.79:0/3994923265 conn(0x7fb8fc071c30 msgr2=0x7fb8fc0b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:51.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 shutdown_connections 2024-04-26T19:50:51.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.025+0000 7fb9015af640 1 -- 172.21.15.79:0/3994923265 wait complete. 2024-04-26T19:50:51.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T19:50:51.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.135+0000 7f618a047640 1 Processor -- start 2024-04-26T19:50:51.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.136+0000 7f618a047640 1 -- start start 2024-04-26T19:50:51.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.136+0000 7f618a047640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f618406f920 0x7f61840b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:51.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.136+0000 7f618a047640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61840ba6f0 con 0x7f61841598c0 2024-04-26T19:50:51.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.136+0000 7f618a047640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61840ba410 con 0x7f6184160610 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.136+0000 7f618a047640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f61840d5100 con 0x7f618406f920 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.136+0000 7f61837fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f61841598c0 0x7f61840b9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46202/0 (socket says 172.21.15.79:46202) 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.136+0000 7f61837fe640 1 -- 172.21.15.79:0/3566314502 learned_addr learned my addr 172.21.15.79:0/3566314502 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.137+0000 7f6183fff640 1 --2- 172.21.15.79:0/3566314502 >> v2:172.21.15.83:3300/0 conn(0x7f618406f920 0x7f61840b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.137+0000 7f61827fc640 1 -- 172.21.15.79:0/3566314502 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3605643940 0 0) 0x7f61840ba6f0 con 0x7f61841598c0 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.137+0000 7f61827fc640 1 -- 172.21.15.79:0/3566314502 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f616c003520 con 0x7f61841598c0 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.137+0000 7f61827fc640 1 -- 172.21.15.79:0/3566314502 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2140527884 0 0) 0x7f61840ba410 con 0x7f6184160610 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.137+0000 7f61827fc640 1 -- 172.21.15.79:0/3566314502 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61840ba6f0 con 0x7f6184160610 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.137+0000 7f61827fc640 1 -- 172.21.15.79:0/3566314502 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2914818356 0 0) 0x7f616c003520 con 0x7f61841598c0 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.137+0000 7f6183fff640 1 -- 172.21.15.79:0/3566314502 >> v1:172.21.15.83:6789/0 conn(0x7f6184160610 legacy=0x7f618406f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.137+0000 7f6183fff640 1 -- 172.21.15.79:0/3566314502 >> v1:172.21.15.79:6789/0 conn(0x7f61841598c0 legacy=0x7f61840b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.137+0000 7f6183fff640 1 -- 172.21.15.79:0/3566314502 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61840b8660 con 0x7f618406f920 2024-04-26T19:50:51.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f6183fff640 1 --2- 172.21.15.79:0/3566314502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618406f920 0x7f61840b9140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61840d5280 tx=0x7f617c02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79a6c7a90262f6e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:51.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f61827fc640 1 -- 172.21.15.79:0/3566314502 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6174001670 con 0x7f618406f920 2024-04-26T19:50:51.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f61827fc640 1 -- 172.21.15.79:0/3566314502 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617c007540 con 0x7f618406f920 2024-04-26T19:50:51.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f61827fc640 1 -- 172.21.15.79:0/3566314502 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c0078f0 con 0x7f618406f920 2024-04-26T19:50:51.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f618a047640 1 -- 172.21.15.79:0/3566314502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618406f920 msgr2=0x7f61840b9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:51.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f618a047640 1 --2- 172.21.15.79:0/3566314502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618406f920 0x7f61840b9140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f61840d5280 tx=0x7f617c02a5f0 comp rx=0 tx=0).stop 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f618a047640 1 -- 172.21.15.79:0/3566314502 shutdown_connections 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f618a047640 1 --2- 172.21.15.79:0/3566314502 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618406f920 0x7f61840b9140 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f618a047640 1 -- 172.21.15.79:0/3566314502 >> 172.21.15.79:0/3566314502 conn(0x7f61840b4930 msgr2=0x7f61840b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f618a047640 1 -- 172.21.15.79:0/3566314502 shutdown_connections 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.138+0000 7f618a047640 1 -- 172.21.15.79:0/3566314502 wait complete. 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.140+0000 7f618a047640 1 Processor -- start 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.140+0000 7f618a047640 1 -- start start 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.140+0000 7f618a047640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618406f920 0x7f61841f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.140+0000 7f618a047640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61841598c0 0x7f61841576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.140+0000 7f618a047640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6184160610 0x7f61840d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.140+0000 7f618a047640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6184159e20 con 0x7f6184160610 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.140+0000 7f618a047640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6184159ca0 con 0x7f618406f920 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.140+0000 7f618a047640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f61840b3920 con 0x7f61841598c0 2024-04-26T19:50:51.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.140+0000 7f6183fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6184160610 0x7f61840d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f6183fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6184160610 0x7f61840d33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38614/0 (socket says 172.21.15.79:38614) 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f6183fff640 1 -- 172.21.15.79:0/4291637920 learned_addr learned my addr 172.21.15.79:0/4291637920 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f61837fe640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618406f920 0x7f61841f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f6182ffd640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61841598c0 0x7f61841576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f6183fff640 1 -- 172.21.15.79:0/4291637920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61841598c0 msgr2=0x7f61841576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f6183fff640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61841598c0 0x7f61841576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f6183fff640 1 -- 172.21.15.79:0/4291637920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618406f920 msgr2=0x7f61841f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f6183fff640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618406f920 0x7f61841f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f6183fff640 1 -- 172.21.15.79:0/4291637920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61840aeb20 con 0x7f6184160610 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f6183fff640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6184160610 0x7f61840d33f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f617c009e50 tx=0x7f617c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f616bfff640 1 -- 172.21.15.79:0/4291637920 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c040070 con 0x7f6184160610 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f616bfff640 1 -- 172.21.15.79:0/4291637920 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f617c031c70 con 0x7f6184160610 2024-04-26T19:50:51.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61840aee00 con 0x7f6184160610 2024-04-26T19:50:51.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f616bfff640 1 -- 172.21.15.79:0/4291637920 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c009450 con 0x7f6184160610 2024-04-26T19:50:51.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.141+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61840af3c0 con 0x7f6184160610 2024-04-26T19:50:51.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.142+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61840506c0 con 0x7f6184160610 2024-04-26T19:50:51.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.143+0000 7f616bfff640 1 -- 172.21.15.79:0/4291637920 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f617c039040 con 0x7f6184160610 2024-04-26T19:50:51.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.144+0000 7f616bfff640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f617006c8f0 0x7f617006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:51.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.144+0000 7f61837fe640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f617006c8f0 0x7f617006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:51.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.144+0000 7f616bfff640 1 -- 172.21.15.79:0/4291637920 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f617c0acc30 con 0x7f6184160610 2024-04-26T19:50:51.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.144+0000 7f61837fe640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f617006c8f0 0x7f617006edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f61740017e0 tx=0x7f6174001610 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:51.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.149+0000 7f616bfff640 1 -- 172.21.15.79:0/4291637920 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f617c031e20 con 0x7f6184160610 2024-04-26T19:50:51.365 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:51.365+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6184157c20 con 0x7f6184160610 2024-04-26T19:50:52.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.169+0000 7f616bfff640 1 -- 172.21.15.79:0/4291637920 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7f617c07a650 con 0x7f6184160610 2024-04-26T19:50:52.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f617006c8f0 msgr2=0x7f617006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f617006c8f0 0x7f617006edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f61740017e0 tx=0x7f6174001610 comp rx=0 tx=0).stop 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6184160610 msgr2=0x7f61840d33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6184160610 0x7f61840d33f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f617c009e50 tx=0x7f617c004a40 comp rx=0 tx=0).stop 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 shutdown_connections 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f617006c8f0 0x7f617006edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6184160610 0x7f61840d33f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61841598c0 0x7f61841576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 --2- 172.21.15.79:0/4291637920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f618406f920 0x7f61841f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 >> 172.21.15.79:0/4291637920 conn(0x7f61840b4930 msgr2=0x7f6184072e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 shutdown_connections 2024-04-26T19:50:52.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.172+0000 7f618a047640 1 -- 172.21.15.79:0/4291637920 wait complete. 2024-04-26T19:50:52.185 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-26T19:50:52.185 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T19:50:52.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.289+0000 7f8605e7b640 1 Processor -- start 2024-04-26T19:50:52.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.290+0000 7f8605e7b640 1 -- start start 2024-04-26T19:50:52.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.290+0000 7f8605e7b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8600162b30 0x7f86000b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:52.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.290+0000 7f8605e7b640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86000b72e0 con 0x7f860006d790 2024-04-26T19:50:52.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.290+0000 7f8605e7b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f860006d360 con 0x7f86000b1e40 2024-04-26T19:50:52.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.290+0000 7f8605e7b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8600162f10 con 0x7f8600162b30 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.290+0000 7f85feffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f860006d790 0x7f86000b64c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46212/0 (socket says 172.21.15.79:46212) 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85feffd640 1 -- 172.21.15.79:0/1821961720 learned_addr learned my addr 172.21.15.79:0/1821961720 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85ff7fe640 1 --2- 172.21.15.79:0/1821961720 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8600162b30 0x7f86000b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85fe7fc640 1 -- 172.21.15.79:0/1821961720 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2983191039 0 0) 0x7f86000b72e0 con 0x7f860006d790 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85fe7fc640 1 -- 172.21.15.79:0/1821961720 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f85e8003520 con 0x7f860006d790 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85fe7fc640 1 -- 172.21.15.79:0/1821961720 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2190861138 0 0) 0x7f860006d360 con 0x7f86000b1e40 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85fe7fc640 1 -- 172.21.15.79:0/1821961720 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f86000b72e0 con 0x7f86000b1e40 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85ff7fe640 1 -- 172.21.15.79:0/1821961720 >> v1:172.21.15.83:6789/0 conn(0x7f86000b1e40 legacy=0x7f86000b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85ff7fe640 1 -- 172.21.15.79:0/1821961720 >> v1:172.21.15.79:6789/0 conn(0x7f860006d790 legacy=0x7f86000b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85fe7fc640 1 -- 172.21.15.79:0/1821961720 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2292862365 0 0) 0x7f85e8003520 con 0x7f860006d790 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.291+0000 7f85ff7fe640 1 -- 172.21.15.79:0/1821961720 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8600163370 con 0x7f8600162b30 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f85ff7fe640 1 --2- 172.21.15.79:0/1821961720 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8600162b30 0x7f86000b3480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85f00097d0 tx=0x7f85f002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c71a36a7778edfa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f85fe7fc640 1 -- 172.21.15.79:0/1821961720 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0037070 con 0x7f8600162b30 2024-04-26T19:50:52.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f85fe7fc640 1 -- 172.21.15.79:0/1821961720 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f002ab10 con 0x7f8600162b30 2024-04-26T19:50:52.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f85fe7fc640 1 -- 172.21.15.79:0/1821961720 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4001670 con 0x7f8600162b30 2024-04-26T19:50:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f8605e7b640 1 -- 172.21.15.79:0/1821961720 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8600162b30 msgr2=0x7f86000b3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f8605e7b640 1 --2- 172.21.15.79:0/1821961720 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8600162b30 0x7f86000b3480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85f00097d0 tx=0x7f85f002a080 comp rx=0 tx=0).stop 2024-04-26T19:50:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f8605e7b640 1 -- 172.21.15.79:0/1821961720 shutdown_connections 2024-04-26T19:50:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f8605e7b640 1 --2- 172.21.15.79:0/1821961720 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8600162b30 0x7f86000b3480 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f8605e7b640 1 -- 172.21.15.79:0/1821961720 >> 172.21.15.79:0/1821961720 conn(0x7f86000b4500 msgr2=0x7f86000b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f8605e7b640 1 -- 172.21.15.79:0/1821961720 shutdown_connections 2024-04-26T19:50:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.292+0000 7f8605e7b640 1 -- 172.21.15.79:0/1821961720 wait complete. 2024-04-26T19:50:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.294+0000 7f8605e7b640 1 Processor -- start 2024-04-26T19:50:52.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.294+0000 7f8605e7b640 1 -- start start 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.294+0000 7f8605e7b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f860006d790 0x7f8600152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.294+0000 7f8605e7b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f86000b1e40 0x7f86000ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.294+0000 7f8605e7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600162b30 0x7f86000cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.294+0000 7f8605e7b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8600163930 con 0x7f860006d790 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.294+0000 7f8605e7b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8600163120 con 0x7f8600162b30 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.294+0000 7f8605e7b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8600163c10 con 0x7f86000b1e40 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85ff7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f860006d790 0x7f8600152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85ff7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f860006d790 0x7f8600152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38622/0 (socket says 172.21.15.79:38622) 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85ff7fe640 1 -- 172.21.15.79:0/804628994 learned_addr learned my addr 172.21.15.79:0/804628994 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85fffff640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600162b30 0x7f86000cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85feffd640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f86000b1e40 0x7f86000ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85ff7fe640 1 -- 172.21.15.79:0/804628994 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f86000b1e40 msgr2=0x7f86000ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85ff7fe640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f86000b1e40 0x7f86000ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:52.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85ff7fe640 1 -- 172.21.15.79:0/804628994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600162b30 msgr2=0x7f86000cee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85ff7fe640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600162b30 0x7f86000cee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85ff7fe640 1 -- 172.21.15.79:0/804628994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86001f86e0 con 0x7f860006d790 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85ff7fe640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f860006d790 0x7f8600152e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f85f00097d0 tx=0x7f85f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85e7fff640 1 -- 172.21.15.79:0/804628994 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0042020 con 0x7f860006d790 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85e7fff640 1 -- 172.21.15.79:0/804628994 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85f003d670 con 0x7f860006d790 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86001f8980 con 0x7f860006d790 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f85e7fff640 1 -- 172.21.15.79:0/804628994 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0037040 con 0x7f860006d790 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.295+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86001f8f40 con 0x7f860006d790 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.296+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86000506c0 con 0x7f860006d790 2024-04-26T19:50:52.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.297+0000 7f85e7fff640 1 -- 172.21.15.79:0/804628994 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85f0045020 con 0x7f860006d790 2024-04-26T19:50:52.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.297+0000 7f85e7fff640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f85ec06c8f0 0x7f85ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:52.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.297+0000 7f85e7fff640 1 -- 172.21.15.79:0/804628994 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f85f00ac580 con 0x7f860006d790 2024-04-26T19:50:52.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.297+0000 7f85feffd640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f85ec06c8f0 0x7f85ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:52.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.298+0000 7f85feffd640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f85ec06c8f0 0x7f85ec06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f86000b2550 tx=0x7f85f4001610 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:52.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.301+0000 7f85e7fff640 1 -- 172.21.15.79:0/804628994 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f85f004c030 con 0x7f860006d790 2024-04-26T19:50:52.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:52.515+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f860015ceb0 con 0x7f860006d790 2024-04-26T19:50:53.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.179+0000 7f85e7fff640 1 -- 172.21.15.79:0/804628994 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f85f007a0b0 con 0x7f860006d790 2024-04-26T19:50:53.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.181+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f85ec06c8f0 msgr2=0x7f85ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.181+0000 7f8605e7b640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f85ec06c8f0 0x7f85ec06edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f86000b2550 tx=0x7f85f4001610 comp rx=0 tx=0).stop 2024-04-26T19:50:53.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.181+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f860006d790 msgr2=0x7f8600152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.181+0000 7f8605e7b640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f860006d790 0x7f8600152e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f85f00097d0 tx=0x7f85f00046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.181+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 shutdown_connections 2024-04-26T19:50:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.181+0000 7f8605e7b640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f85ec06c8f0 0x7f85ec06edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.181+0000 7f8605e7b640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8600162b30 0x7f86000cee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.181+0000 7f8605e7b640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f86000b1e40 0x7f86000ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.181+0000 7f8605e7b640 1 --2- 172.21.15.79:0/804628994 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f860006d790 0x7f8600152e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.182+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 >> 172.21.15.79:0/804628994 conn(0x7f86000b4500 msgr2=0x7f86000b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.182+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 shutdown_connections 2024-04-26T19:50:53.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.182+0000 7f8605e7b640 1 -- 172.21.15.79:0/804628994 wait complete. 2024-04-26T19:50:53.196 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T19:50:53.196 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:50:53.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.298+0000 7f750a71c640 1 Processor -- start 2024-04-26T19:50:53.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.298+0000 7f750a71c640 1 -- start start 2024-04-26T19:50:53.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f750a71c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f750406fb20 0x7f75040d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f750a71c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f750406ff00 0x7f75040d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f750a71c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75040b1290 con 0x7f75041488c0 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f750a71c640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f75040d3e60 con 0x7f750406ff00 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f750a71c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f75040d4090 con 0x7f750406fb20 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f75037fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f750406ff00 0x7f75040d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f75037fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f750406ff00 0x7f75040d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38636/0 (socket says 172.21.15.79:38636) 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f75037fe640 1 -- 172.21.15.79:0/2564504063 learned_addr learned my addr 172.21.15.79:0/2564504063 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f75037fe640 1 -- 172.21.15.79:0/2564504063 >> v2:172.21.15.83:3300/0 conn(0x7f750406fb20 msgr2=0x7f75040d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f75037fe640 1 --2- 172.21.15.79:0/2564504063 >> v2:172.21.15.83:3300/0 conn(0x7f750406fb20 0x7f75040d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f75037fe640 1 -- 172.21.15.79:0/2564504063 >> v1:172.21.15.83:6789/0 conn(0x7f75041488c0 legacy=0x7f7504148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.299+0000 7f75037fe640 1 -- 172.21.15.79:0/2564504063 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75040d4370 con 0x7f750406ff00 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.300+0000 7f75037fe640 1 --2- 172.21.15.79:0/2564504063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f750406ff00 0x7f75040d3920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f74f80098a0 tx=0x7f74f802a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e1efadb0f71376 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.300+0000 7f7502ffd640 1 -- 172.21.15.79:0/2564504063 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8037070 con 0x7f750406ff00 2024-04-26T19:50:53.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.300+0000 7f7502ffd640 1 -- 172.21.15.79:0/2564504063 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74f8032640 con 0x7f750406ff00 2024-04-26T19:50:53.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.300+0000 7f7502ffd640 1 -- 172.21.15.79:0/2564504063 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f8032a10 con 0x7f750406ff00 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.301+0000 7f750a71c640 1 -- 172.21.15.79:0/2564504063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f750406ff00 msgr2=0x7f75040d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.301+0000 7f750a71c640 1 --2- 172.21.15.79:0/2564504063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f750406ff00 0x7f75040d3920 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f74f80098a0 tx=0x7f74f802a320 comp rx=0 tx=0).stop 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.301+0000 7f750a71c640 1 -- 172.21.15.79:0/2564504063 shutdown_connections 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.301+0000 7f750a71c640 1 --2- 172.21.15.79:0/2564504063 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f750406ff00 0x7f75040d3920 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.301+0000 7f750a71c640 1 --2- 172.21.15.79:0/2564504063 >> v2:172.21.15.83:3300/0 conn(0x7f750406fb20 0x7f75040d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.301+0000 7f750a71c640 1 -- 172.21.15.79:0/2564504063 >> 172.21.15.79:0/2564504063 conn(0x7f7504071c30 msgr2=0x7f750406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.301+0000 7f750a71c640 1 -- 172.21.15.79:0/2564504063 shutdown_connections 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.301+0000 7f750a71c640 1 -- 172.21.15.79:0/2564504063 wait complete. 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.302+0000 7f750a71c640 1 Processor -- start 2024-04-26T19:50:53.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.302+0000 7f750a71c640 1 -- start start 2024-04-26T19:50:53.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f750a71c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f750406fb20 0x7f75041625c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f750a71c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750406ff00 0x7f7504141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f750a71c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75041488c0 0x7f75040c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f750a71c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f75040d4c10 con 0x7f75041488c0 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f750a71c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75040d4930 con 0x7f750406ff00 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f750a71c640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f75040aebb0 con 0x7f750406fb20 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7508cb9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75041488c0 0x7f75040c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7508cb9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75041488c0 0x7f75040c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38640/0 (socket says 172.21.15.79:38640) 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7508cb9640 1 -- 172.21.15.79:0/986783677 learned_addr learned my addr 172.21.15.79:0/986783677 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f75037fe640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750406ff00 0x7f7504141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7503fff640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f750406fb20 0x7f75041625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7508cb9640 1 -- 172.21.15.79:0/986783677 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f750406fb20 msgr2=0x7f75041625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7508cb9640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f750406fb20 0x7f75041625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7508cb9640 1 -- 172.21.15.79:0/986783677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750406ff00 msgr2=0x7f7504141ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7508cb9640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750406ff00 0x7f7504141ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7508cb9640 1 -- 172.21.15.79:0/986783677 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75040c5bc0 con 0x7f75041488c0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7503fff640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f750406fb20 0x7f75041625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.303+0000 7f7508cb9640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75041488c0 0x7f75040c6c00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00b9c0 tx=0x7f74fc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.304+0000 7f7500ff9640 1 -- 172.21.15.79:0/986783677 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc019070 con 0x7f75041488c0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.304+0000 7f7500ff9640 1 -- 172.21.15.79:0/986783677 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74fc00cba0 con 0x7f75041488c0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.304+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75040c5ea0 con 0x7f75041488c0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.304+0000 7f7500ff9640 1 -- 172.21.15.79:0/986783677 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74fc014440 con 0x7f75041488c0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.304+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7504162b00 con 0x7f75041488c0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.304+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75040506c0 con 0x7f75041488c0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.305+0000 7f7500ff9640 1 -- 172.21.15.79:0/986783677 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f74fc01c070 con 0x7f75041488c0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.305+0000 7f7500ff9640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f74f006c8f0 0x7f74f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.305+0000 7f7500ff9640 1 -- 172.21.15.79:0/986783677 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f74fc08baf0 con 0x7f75041488c0 2024-04-26T19:50:53.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.305+0000 7f7503fff640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f74f006c8f0 0x7f74f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.306+0000 7f7503fff640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f74f006c8f0 0x7f74f006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009930 tx=0x7f74f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:53.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.310+0000 7f7500ff9640 1 -- 172.21.15.79:0/986783677 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f74fc059620 con 0x7f75041488c0 2024-04-26T19:50:53.521 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.520+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7504072950 con 0x7f75041488c0 2024-04-26T19:50:53.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.521+0000 7f7500ff9640 1 -- 172.21.15.79:0/986783677 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f74fc05d920 con 0x7f75041488c0 2024-04-26T19:50:53.522 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:53.522 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:52.517190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:50:53.522 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f74f006c8f0 msgr2=0x7f74f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f74f006c8f0 0x7f74f006edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f74f4009930 tx=0x7f74f4008040 comp rx=0 tx=0).stop 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75041488c0 msgr2=0x7f75040c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75041488c0 0x7f75040c6c00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00b9c0 tx=0x7f74fc00be80 comp rx=0 tx=0).stop 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 shutdown_connections 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f74f006c8f0 0x7f74f006edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75041488c0 0x7f75040c6c00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f750406ff00 0x7f7504141ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 --2- 172.21.15.79:0/986783677 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f750406fb20 0x7f75041625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.524 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 >> 172.21.15.79:0/986783677 conn(0x7f7504071c30 msgr2=0x7f7504146300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:53.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 shutdown_connections 2024-04-26T19:50:53.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.523+0000 7f750a71c640 1 -- 172.21.15.79:0/986783677 wait complete. 2024-04-26T19:50:53.535 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:53.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.630+0000 7fc4d9db4640 1 Processor -- start 2024-04-26T19:50:53.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.630+0000 7fc4d9db4640 1 -- start start 2024-04-26T19:50:53.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.631+0000 7fc4d9db4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160770 0x7fc4d41637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.631+0000 7fc4d9db4640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d4144ca0 con 0x7fc4d4160390 2024-04-26T19:50:53.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.631+0000 7fc4d9db4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d4163e00 con 0x7fc4d40b4220 2024-04-26T19:50:53.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.631+0000 7fc4d9db4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d4144f30 con 0x7fc4d4160770 2024-04-26T19:50:53.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.631+0000 7fc4d37fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc4d4160390 0x7fc4d40b3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46224/0 (socket says 172.21.15.79:46224) 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.631+0000 7fc4d37fe640 1 -- 172.21.15.79:0/3049414211 learned_addr learned my addr 172.21.15.79:0/3049414211 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.631+0000 7fc4d3fff640 1 --2- 172.21.15.79:0/3049414211 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160770 0x7fc4d41637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.631+0000 7fc4d27fc640 1 -- 172.21.15.79:0/3049414211 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3359261153 0 0) 0x7fc4d4144ca0 con 0x7fc4d4160390 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.631+0000 7fc4d27fc640 1 -- 172.21.15.79:0/3049414211 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4bc003520 con 0x7fc4d4160390 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.632+0000 7fc4d27fc640 1 -- 172.21.15.79:0/3049414211 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3750631405 0 0) 0x7fc4d4163e00 con 0x7fc4d40b4220 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.632+0000 7fc4d27fc640 1 -- 172.21.15.79:0/3049414211 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4d4144ca0 con 0x7fc4d40b4220 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.632+0000 7fc4d3fff640 1 -- 172.21.15.79:0/3049414211 >> v1:172.21.15.83:6789/0 conn(0x7fc4d40b4220 legacy=0x7fc4d41630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.632+0000 7fc4d27fc640 1 -- 172.21.15.79:0/3049414211 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1639874238 0 0) 0x7fc4bc003520 con 0x7fc4d4160390 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.632+0000 7fc4d3fff640 1 -- 172.21.15.79:0/3049414211 >> v1:172.21.15.79:6789/0 conn(0x7fc4d4160390 legacy=0x7fc4d40b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.632+0000 7fc4d3fff640 1 -- 172.21.15.79:0/3049414211 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d4145210 con 0x7fc4d4160770 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.632+0000 7fc4d3fff640 1 --2- 172.21.15.79:0/3049414211 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160770 0x7fc4d41637e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009dc0 tx=0x7fc4cc02a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc2987dba489a7cb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:53.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d27fc640 1 -- 172.21.15.79:0/3049414211 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4001670 con 0x7fc4d4160770 2024-04-26T19:50:53.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d27fc640 1 -- 172.21.15.79:0/3049414211 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4cc007540 con 0x7fc4d4160770 2024-04-26T19:50:53.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d27fc640 1 -- 172.21.15.79:0/3049414211 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4cc0078f0 con 0x7fc4d4160770 2024-04-26T19:50:53.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d9db4640 1 -- 172.21.15.79:0/3049414211 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160770 msgr2=0x7fc4d41637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d9db4640 1 --2- 172.21.15.79:0/3049414211 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160770 0x7fc4d41637e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009dc0 tx=0x7fc4cc02a5d0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d9db4640 1 -- 172.21.15.79:0/3049414211 shutdown_connections 2024-04-26T19:50:53.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d9db4640 1 --2- 172.21.15.79:0/3049414211 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160770 0x7fc4d41637e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d9db4640 1 -- 172.21.15.79:0/3049414211 >> 172.21.15.79:0/3049414211 conn(0x7fc4d40b2720 msgr2=0x7fc4d4070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:53.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d9db4640 1 -- 172.21.15.79:0/3049414211 shutdown_connections 2024-04-26T19:50:53.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.633+0000 7fc4d9db4640 1 -- 172.21.15.79:0/3049414211 wait complete. 2024-04-26T19:50:53.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.634+0000 7fc4d9db4640 1 Processor -- start 2024-04-26T19:50:53.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.634+0000 7fc4d9db4640 1 -- start start 2024-04-26T19:50:53.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d9db4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4d40b4220 0x7fc4d4145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d9db4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160390 0x7fc4d406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d9db4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d4160770 0x7fc4d4141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d9db4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4145b90 con 0x7fc4d40b4220 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d9db4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d4145a10 con 0x7fc4d4160770 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d9db4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d416a470 con 0x7fc4d4160390 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4d40b4220 0x7fc4d4145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4d40b4220 0x7fc4d4145da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38652/0 (socket says 172.21.15.79:38652) 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d37fe640 1 -- 172.21.15.79:0/2832708920 learned_addr learned my addr 172.21.15.79:0/2832708920 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d2ffd640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160390 0x7fc4d406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d3fff640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d4160770 0x7fc4d4141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d37fe640 1 -- 172.21.15.79:0/2832708920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160390 msgr2=0x7fc4d406f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d37fe640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160390 0x7fc4d406f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d37fe640 1 -- 172.21.15.79:0/2832708920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d4160770 msgr2=0x7fc4d4141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d37fe640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d4160770 0x7fc4d4141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.635+0000 7fc4d37fe640 1 -- 172.21.15.79:0/2832708920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d413cda0 con 0x7fc4d40b4220 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.636+0000 7fc4d37fe640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4d40b4220 0x7fc4d4145da0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c5a0 tx=0x7fc4c400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.636+0000 7fc4bbfff640 1 -- 172.21.15.79:0/2832708920 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4017070 con 0x7fc4d40b4220 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.636+0000 7fc4bbfff640 1 -- 172.21.15.79:0/2832708920 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c4005ce0 con 0x7fc4d40b4220 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.636+0000 7fc4bbfff640 1 -- 172.21.15.79:0/2832708920 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c40072b0 con 0x7fc4d40b4220 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.636+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d413d080 con 0x7fc4d40b4220 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.636+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d413d640 con 0x7fc4d40b4220 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.637+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d40506c0 con 0x7fc4d40b4220 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.637+0000 7fc4bbfff640 1 -- 172.21.15.79:0/2832708920 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc4c401a070 con 0x7fc4d40b4220 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.638+0000 7fc4bbfff640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc4c006c8f0 0x7fc4c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.638+0000 7fc4d2ffd640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc4c006c8f0 0x7fc4c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.638+0000 7fc4bbfff640 1 -- 172.21.15.79:0/2832708920 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc4c408b3a0 con 0x7fc4d40b4220 2024-04-26T19:50:53.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.638+0000 7fc4d2ffd640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc4c006c8f0 0x7fc4c006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40d21f0 tx=0x7fc4c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:53.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.642+0000 7fc4bbfff640 1 -- 172.21.15.79:0/2832708920 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc4c4059490 con 0x7fc4d40b4220 2024-04-26T19:50:53.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.817+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4d406d0f0 con 0x7fc4d40b4220 2024-04-26T19:50:53.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.819+0000 7fc4bbfff640 1 -- 172.21.15.79:0/2832708920 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fc4c4058d90 con 0x7fc4d40b4220 2024-04-26T19:50:53.819 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:53.820 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:53.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc4c006c8f0 msgr2=0x7fc4c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc4c006c8f0 0x7fc4c006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40d21f0 tx=0x7fc4c8009040 comp rx=0 tx=0).stop 2024-04-26T19:50:53.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4d40b4220 msgr2=0x7fc4d4145da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4d40b4220 0x7fc4d4145da0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c5a0 tx=0x7fc4c400ca60 comp rx=0 tx=0).stop 2024-04-26T19:50:53.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 shutdown_connections 2024-04-26T19:50:53.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc4c006c8f0 0x7fc4c006edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc4d4160770 0x7fc4d4141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc4d4160390 0x7fc4d406f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 --2- 172.21.15.79:0/2832708920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4d40b4220 0x7fc4d4145da0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 >> 172.21.15.79:0/2832708920 conn(0x7fc4d40b2720 msgr2=0x7fc4d4161fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:53.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 shutdown_connections 2024-04-26T19:50:53.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.821+0000 7fc4d9db4640 1 -- 172.21.15.79:0/2832708920 wait complete. 2024-04-26T19:50:53.832 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T19:50:53.833 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:50:53.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.929+0000 7f7daaa23640 1 Processor -- start 2024-04-26T19:50:53.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.929+0000 7f7daaa23640 1 -- start start 2024-04-26T19:50:53.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.929+0000 7f7daaa23640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7da415f2e0 0x7f7da40700d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.929+0000 7f7daaa23640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7da4070740 con 0x7f7da40b64f0 2024-04-26T19:50:53.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.929+0000 7f7daaa23640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7da40ce5d0 con 0x7f7da40cf0b0 2024-04-26T19:50:53.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.929+0000 7f7daaa23640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f7da40cec20 con 0x7f7da415f2e0 2024-04-26T19:50:53.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.930+0000 7f7da3fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7da415f2e0 0x7f7da40700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.930+0000 7f7da8fc0640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f7da40b64f0 0x7f7da40cb3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46226/0 (socket says 172.21.15.79:46226) 2024-04-26T19:50:53.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.930+0000 7f7da8fc0640 1 -- 172.21.15.79:0/2943721678 learned_addr learned my addr 172.21.15.79:0/2943721678 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:53.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.930+0000 7f7da3fff640 1 -- 172.21.15.79:0/2943721678 >> v1:172.21.15.83:6789/0 conn(0x7f7da40cf0b0 legacy=0x7f7da40d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:53.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.930+0000 7f7da3fff640 1 -- 172.21.15.79:0/2943721678 >> v1:172.21.15.79:6789/0 conn(0x7f7da40b64f0 legacy=0x7f7da40cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.930+0000 7f7da2ffd640 1 -- 172.21.15.79:0/2943721678 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1173769986 0 0) 0x7f7da4070740 con 0x7f7da40b64f0 2024-04-26T19:50:53.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.930+0000 7f7da3fff640 1 -- 172.21.15.79:0/2943721678 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da40ce8b0 con 0x7f7da415f2e0 2024-04-26T19:50:53.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7da3fff640 1 --2- 172.21.15.79:0/2943721678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 0x7f7da40700d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7da40ceda0 tx=0x7f7d9402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71bdf7a916e081b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:53.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7da2ffd640 1 -- 172.21.15.79:0/2943721678 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94037070 con 0x7f7da415f2e0 2024-04-26T19:50:53.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7da2ffd640 1 -- 172.21.15.79:0/2943721678 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d94032440 con 0x7f7da415f2e0 2024-04-26T19:50:53.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7da2ffd640 1 -- 172.21.15.79:0/2943721678 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94032810 con 0x7f7da415f2e0 2024-04-26T19:50:53.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7daaa23640 1 -- 172.21.15.79:0/2943721678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 msgr2=0x7f7da40700d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7daaa23640 1 --2- 172.21.15.79:0/2943721678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 0x7f7da40700d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7da40ceda0 tx=0x7f7d9402a180 comp rx=0 tx=0).stop 2024-04-26T19:50:53.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7daaa23640 1 -- 172.21.15.79:0/2943721678 shutdown_connections 2024-04-26T19:50:53.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7daaa23640 1 --2- 172.21.15.79:0/2943721678 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 0x7f7da40700d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7daaa23640 1 -- 172.21.15.79:0/2943721678 >> 172.21.15.79:0/2943721678 conn(0x7f7da406c770 msgr2=0x7f7da406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:53.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7daaa23640 1 -- 172.21.15.79:0/2943721678 shutdown_connections 2024-04-26T19:50:53.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.931+0000 7f7daaa23640 1 -- 172.21.15.79:0/2943721678 wait complete. 2024-04-26T19:50:53.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.932+0000 7f7daaa23640 1 Processor -- start 2024-04-26T19:50:53.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7daaa23640 1 -- start start 2024-04-26T19:50:53.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7daaa23640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7da40b64f0 0x7f7da41528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7daaa23640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7da40cf0b0 0x7f7da4157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7daaa23640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 0x7f7da415deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7daaa23640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7da415f6c0 con 0x7f7da415f2e0 2024-04-26T19:50:53.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7daaa23640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7da40cf510 con 0x7f7da40b64f0 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7daaa23640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f7da40cbac0 con 0x7f7da40cf0b0 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7da8fc0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 0x7f7da415deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7da8fc0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 0x7f7da415deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38662/0 (socket says 172.21.15.79:38662) 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7da8fc0640 1 -- 172.21.15.79:0/3380352639 learned_addr learned my addr 172.21.15.79:0/3380352639 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.933+0000 7f7da3fff640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7da40b64f0 0x7f7da41528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da37fe640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7da40cf0b0 0x7f7da4157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da8fc0640 1 -- 172.21.15.79:0/3380352639 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7da40cf0b0 msgr2=0x7f7da4157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da8fc0640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7da40cf0b0 0x7f7da4157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da8fc0640 1 -- 172.21.15.79:0/3380352639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7da40b64f0 msgr2=0x7f7da41528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da8fc0640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7da40b64f0 0x7f7da41528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:53.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da8fc0640 1 -- 172.21.15.79:0/3380352639 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da41f8690 con 0x7f7da415f2e0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da8fc0640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 0x7f7da415deb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c7a0 tx=0x7f7d9c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da0ff9640 1 -- 172.21.15.79:0/3380352639 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c018070 con 0x7f7da415f2e0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da0ff9640 1 -- 172.21.15.79:0/3380352639 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d9c00a640 con 0x7f7da415f2e0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da41f8930 con 0x7f7da415f2e0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7da0ff9640 1 -- 172.21.15.79:0/3380352639 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00aab0 con 0x7f7da415f2e0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.934+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41f8ef0 con 0x7f7da415f2e0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.935+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da40506c0 con 0x7f7da415f2e0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.936+0000 7f7da0ff9640 1 -- 172.21.15.79:0/3380352639 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d9c01b070 con 0x7f7da415f2e0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.936+0000 7f7da0ff9640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7d9006c8f0 0x7f7d9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.936+0000 7f7da0ff9640 1 -- 172.21.15.79:0/3380352639 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7d9c08b410 con 0x7f7da415f2e0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.936+0000 7f7da3fff640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7d9006c8f0 0x7f7d9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:53.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.936+0000 7f7da3fff640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7d9006c8f0 0x7f7d9006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d94034000 tx=0x7f7d9402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:53.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:53.940+0000 7f7da0ff9640 1 -- 172.21.15.79:0/3380352639 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7d9c058eb0 con 0x7f7da415f2e0 2024-04-26T19:50:54.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.112+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7da40730d0 con 0x7f7d9006c8f0 2024-04-26T19:50:54.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.121+0000 7f7da0ff9640 1 -- 172.21.15.79:0/3380352639 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f7da40730d0 con 0x7f7d9006c8f0 2024-04-26T19:50:54.127 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:54.129 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:50:54.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7d9006c8f0 msgr2=0x7f7d9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7d9006c8f0 0x7f7d9006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7d94034000 tx=0x7f7d9402e000 comp rx=0 tx=0).stop 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 msgr2=0x7f7da415deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 0x7f7da415deb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c00c7a0 tx=0x7f7d9c00cc60 comp rx=0 tx=0).stop 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 shutdown_connections 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f7d9006c8f0 0x7f7d9006edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7da415f2e0 0x7f7da415deb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7da40cf0b0 0x7f7da4157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 --2- 172.21.15.79:0/3380352639 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7da40b64f0 0x7f7da41528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 >> 172.21.15.79:0/3380352639 conn(0x7f7da406c770 msgr2=0x7f7da40d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 shutdown_connections 2024-04-26T19:50:54.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.124+0000 7f7daaa23640 1 -- 172.21.15.79:0/3380352639 wait complete. 2024-04-26T19:50:54.136 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-26T19:50:52.256581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219460,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755260,"statfs":{"total":773094113280,"available":772869386240,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002481"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300284+0000","last_change":"2024-04-26T19:50:30.105908+0000","last_active":"2024-04-26T19:50:38.300284+0000","last_peered":"2024-04-26T19:50:38.300284+0000","last_clean":"2024-04-26T19:50:38.300284+0000","last_became_active":"2024-04-26T19:50:30.105650+0000","last_became_peered":"2024-04-26T19:50:30.105650+0000","last_unstale":"2024-04-26T19:50:38.300284+0000","last_undegraded":"2024-04-26T19:50:38.300284+0000","last_fullsized":"2024-04-26T19:50:38.300284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:25.896804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131172+0000","last_change":"2024-04-26T19:50:31.110936+0000","last_active":"2024-04-26T19:50:37.131172+0000","last_peered":"2024-04-26T19:50:37.131172+0000","last_clean":"2024-04-26T19:50:37.131172+0000","last_became_active":"2024-04-26T19:50:31.110602+0000","last_became_peered":"2024-04-26T19:50:31.110602+0000","last_unstale":"2024-04-26T19:50:37.131172+0000","last_undegraded":"2024-04-26T19:50:37.131172+0000","last_fullsized":"2024-04-26T19:50:37.131172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130254+0000","last_change":"2024-04-26T19:50:35.626950+0000","last_active":"2024-04-26T19:50:37.130254+0000","last_peered":"2024-04-26T19:50:37.130254+0000","last_clean":"2024-04-26T19:50:37.130254+0000","last_became_active":"2024-04-26T19:50:35.626309+0000","last_became_peered":"2024-04-26T19:50:35.626309+0000","last_unstale":"2024-04-26T19:50:37.130254+0000","last_undegraded":"2024-04-26T19:50:37.130254+0000","last_fullsized":"2024-04-26T19:50:37.130254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:30:42.055139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131089+0000","last_change":"2024-04-26T19:50:30.112742+0000","last_active":"2024-04-26T19:50:37.131089+0000","last_peered":"2024-04-26T19:50:37.131089+0000","last_clean":"2024-04-26T19:50:37.131089+0000","last_became_active":"2024-04-26T19:50:30.112460+0000","last_became_peered":"2024-04-26T19:50:30.112460+0000","last_unstale":"2024-04-26T19:50:37.131089+0000","last_undegraded":"2024-04-26T19:50:37.131089+0000","last_fullsized":"2024-04-26T19:50:37.131089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:21.153340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300125+0000","last_change":"2024-04-26T19:50:31.114135+0000","last_active":"2024-04-26T19:50:38.300125+0000","last_peered":"2024-04-26T19:50:38.300125+0000","last_clean":"2024-04-26T19:50:38.300125+0000","last_became_active":"2024-04-26T19:50:31.113834+0000","last_became_peered":"2024-04-26T19:50:31.113834+0000","last_unstale":"2024-04-26T19:50:38.300125+0000","last_undegraded":"2024-04-26T19:50:38.300125+0000","last_fullsized":"2024-04-26T19:50:38.300125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129662+0000","last_change":"2024-04-26T19:50:35.141805+0000","last_active":"2024-04-26T19:50:37.129662+0000","last_peered":"2024-04-26T19:50:37.129662+0000","last_clean":"2024-04-26T19:50:37.129662+0000","last_became_active":"2024-04-26T19:50:35.141362+0000","last_became_peered":"2024-04-26T19:50:35.141362+0000","last_unstale":"2024-04-26T19:50:37.129662+0000","last_undegraded":"2024-04-26T19:50:37.129662+0000","last_fullsized":"2024-04-26T19:50:37.129662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:10.325286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653474+0000","last_change":"2024-04-26T19:50:30.107616+0000","last_active":"2024-04-26T19:50:38.653474+0000","last_peered":"2024-04-26T19:50:38.653474+0000","last_clean":"2024-04-26T19:50:38.653474+0000","last_became_active":"2024-04-26T19:50:30.107304+0000","last_became_peered":"2024-04-26T19:50:30.107304+0000","last_unstale":"2024-04-26T19:50:38.653474+0000","last_undegraded":"2024-04-26T19:50:38.653474+0000","last_fullsized":"2024-04-26T19:50:38.653474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:12.144175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652574+0000","last_change":"2024-04-26T19:50:31.114725+0000","last_active":"2024-04-26T19:50:38.652574+0000","last_peered":"2024-04-26T19:50:38.652574+0000","last_clean":"2024-04-26T19:50:38.652574+0000","last_became_active":"2024-04-26T19:50:31.114444+0000","last_became_peered":"2024-04-26T19:50:31.114444+0000","last_unstale":"2024-04-26T19:50:38.652574+0000","last_undegraded":"2024-04-26T19:50:38.652574+0000","last_fullsized":"2024-04-26T19:50:38.652574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129888+0000","last_change":"2024-04-26T19:50:35.145133+0000","last_active":"2024-04-26T19:50:37.129888+0000","last_peered":"2024-04-26T19:50:37.129888+0000","last_clean":"2024-04-26T19:50:37.129888+0000","last_became_active":"2024-04-26T19:50:35.144310+0000","last_became_peered":"2024-04-26T19:50:35.144310+0000","last_unstale":"2024-04-26T19:50:37.129888+0000","last_undegraded":"2024-04-26T19:50:37.129888+0000","last_fullsized":"2024-04-26T19:50:37.129888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:35.220332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652453+0000","last_change":"2024-04-26T19:50:30.115029+0000","last_active":"2024-04-26T19:50:38.652453+0000","last_peered":"2024-04-26T19:50:38.652453+0000","last_clean":"2024-04-26T19:50:38.652453+0000","last_became_active":"2024-04-26T19:50:30.114719+0000","last_became_peered":"2024-04-26T19:50:30.114719+0000","last_unstale":"2024-04-26T19:50:38.652453+0000","last_undegraded":"2024-04-26T19:50:38.652453+0000","last_fullsized":"2024-04-26T19:50:38.652453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:28.557299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654620+0000","last_change":"2024-04-26T19:50:31.114802+0000","last_active":"2024-04-26T19:50:38.654620+0000","last_peered":"2024-04-26T19:50:38.654620+0000","last_clean":"2024-04-26T19:50:38.654620+0000","last_became_active":"2024-04-26T19:50:31.114489+0000","last_became_peered":"2024-04-26T19:50:31.114489+0000","last_unstale":"2024-04-26T19:50:38.654620+0000","last_undegraded":"2024-04-26T19:50:38.654620+0000","last_fullsized":"2024-04-26T19:50:38.654620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129602+0000","last_change":"2024-04-26T19:50:35.147238+0000","last_active":"2024-04-26T19:50:37.129602+0000","last_peered":"2024-04-26T19:50:37.129602+0000","last_clean":"2024-04-26T19:50:37.129602+0000","last_became_active":"2024-04-26T19:50:35.146793+0000","last_became_peered":"2024-04-26T19:50:35.146793+0000","last_unstale":"2024-04-26T19:50:37.129602+0000","last_undegraded":"2024-04-26T19:50:37.129602+0000","last_fullsized":"2024-04-26T19:50:37.129602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:14.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654487+0000","last_change":"2024-04-26T19:50:31.104842+0000","last_active":"2024-04-26T19:50:38.654487+0000","last_peered":"2024-04-26T19:50:38.654487+0000","last_clean":"2024-04-26T19:50:38.654487+0000","last_became_active":"2024-04-26T19:50:31.104587+0000","last_became_peered":"2024-04-26T19:50:31.104587+0000","last_unstale":"2024-04-26T19:50:38.654487+0000","last_undegraded":"2024-04-26T19:50:38.654487+0000","last_fullsized":"2024-04-26T19:50:38.654487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653208+0000","last_change":"2024-04-26T19:50:30.112050+0000","last_active":"2024-04-26T19:50:38.653208+0000","last_peered":"2024-04-26T19:50:38.653208+0000","last_clean":"2024-04-26T19:50:38.653208+0000","last_became_active":"2024-04-26T19:50:30.111754+0000","last_became_peered":"2024-04-26T19:50:30.111754+0000","last_unstale":"2024-04-26T19:50:38.653208+0000","last_undegraded":"2024-04-26T19:50:38.653208+0000","last_fullsized":"2024-04-26T19:50:38.653208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130351+0000","last_change":"2024-04-26T19:50:35.145983+0000","last_active":"2024-04-26T19:50:37.130351+0000","last_peered":"2024-04-26T19:50:37.130351+0000","last_clean":"2024-04-26T19:50:37.130351+0000","last_became_active":"2024-04-26T19:50:35.144815+0000","last_became_peered":"2024-04-26T19:50:35.144815+0000","last_unstale":"2024-04-26T19:50:37.130351+0000","last_undegraded":"2024-04-26T19:50:37.130351+0000","last_fullsized":"2024-04-26T19:50:37.130351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130515+0000","last_change":"2024-04-26T19:50:31.109267+0000","last_active":"2024-04-26T19:50:37.130515+0000","last_peered":"2024-04-26T19:50:37.130515+0000","last_clean":"2024-04-26T19:50:37.130515+0000","last_became_active":"2024-04-26T19:50:31.108771+0000","last_became_peered":"2024-04-26T19:50:31.108771+0000","last_unstale":"2024-04-26T19:50:37.130515+0000","last_undegraded":"2024-04-26T19:50:37.130515+0000","last_fullsized":"2024-04-26T19:50:37.130515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997666+0000","last_change":"2024-04-26T19:50:30.110310+0000","last_active":"2024-04-26T19:50:37.997666+0000","last_peered":"2024-04-26T19:50:37.997666+0000","last_clean":"2024-04-26T19:50:37.997666+0000","last_became_active":"2024-04-26T19:50:30.109819+0000","last_became_peered":"2024-04-26T19:50:30.109819+0000","last_unstale":"2024-04-26T19:50:37.997666+0000","last_undegraded":"2024-04-26T19:50:37.997666+0000","last_fullsized":"2024-04-26T19:50:37.997666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:41.838039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130058+0000","last_change":"2024-04-26T19:50:35.143564+0000","last_active":"2024-04-26T19:50:37.130058+0000","last_peered":"2024-04-26T19:50:37.130058+0000","last_clean":"2024-04-26T19:50:37.130058+0000","last_became_active":"2024-04-26T19:50:35.143119+0000","last_became_peered":"2024-04-26T19:50:35.143119+0000","last_unstale":"2024-04-26T19:50:37.130058+0000","last_undegraded":"2024-04-26T19:50:37.130058+0000","last_fullsized":"2024-04-26T19:50:37.130058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:11.052818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130395+0000","last_change":"2024-04-26T19:50:31.112367+0000","last_active":"2024-04-26T19:50:37.130395+0000","last_peered":"2024-04-26T19:50:37.130395+0000","last_clean":"2024-04-26T19:50:37.130395+0000","last_became_active":"2024-04-26T19:50:31.112070+0000","last_became_peered":"2024-04-26T19:50:31.112070+0000","last_unstale":"2024-04-26T19:50:37.130395+0000","last_undegraded":"2024-04-26T19:50:37.130395+0000","last_fullsized":"2024-04-26T19:50:37.130395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997401+0000","last_change":"2024-04-26T19:50:30.105988+0000","last_active":"2024-04-26T19:50:37.997401+0000","last_peered":"2024-04-26T19:50:37.997401+0000","last_clean":"2024-04-26T19:50:37.997401+0000","last_became_active":"2024-04-26T19:50:30.103503+0000","last_became_peered":"2024-04-26T19:50:30.103503+0000","last_unstale":"2024-04-26T19:50:37.997401+0000","last_undegraded":"2024-04-26T19:50:37.997401+0000","last_fullsized":"2024-04-26T19:50:37.997401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997963+0000","last_change":"2024-04-26T19:50:35.626255+0000","last_active":"2024-04-26T19:50:37.997963+0000","last_peered":"2024-04-26T19:50:37.997963+0000","last_clean":"2024-04-26T19:50:37.997963+0000","last_became_active":"2024-04-26T19:50:35.625845+0000","last_became_peered":"2024-04-26T19:50:35.625845+0000","last_unstale":"2024-04-26T19:50:37.997963+0000","last_undegraded":"2024-04-26T19:50:37.997963+0000","last_fullsized":"2024-04-26T19:50:37.997963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:02.874670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129835+0000","last_change":"2024-04-26T19:50:31.106338+0000","last_active":"2024-04-26T19:50:37.129835+0000","last_peered":"2024-04-26T19:50:37.129835+0000","last_clean":"2024-04-26T19:50:37.129835+0000","last_became_active":"2024-04-26T19:50:31.106097+0000","last_became_peered":"2024-04-26T19:50:31.106097+0000","last_unstale":"2024-04-26T19:50:37.129835+0000","last_undegraded":"2024-04-26T19:50:37.129835+0000","last_fullsized":"2024-04-26T19:50:37.129835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:13.140788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654064+0000","last_change":"2024-04-26T19:50:30.114014+0000","last_active":"2024-04-26T19:50:38.654064+0000","last_peered":"2024-04-26T19:50:38.654064+0000","last_clean":"2024-04-26T19:50:38.654064+0000","last_became_active":"2024-04-26T19:50:30.113720+0000","last_became_peered":"2024-04-26T19:50:30.113720+0000","last_unstale":"2024-04-26T19:50:38.654064+0000","last_undegraded":"2024-04-26T19:50:38.654064+0000","last_fullsized":"2024-04-26T19:50:38.654064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130140+0000","last_change":"2024-04-26T19:50:35.144698+0000","last_active":"2024-04-26T19:50:37.130140+0000","last_peered":"2024-04-26T19:50:37.130140+0000","last_clean":"2024-04-26T19:50:37.130140+0000","last_became_active":"2024-04-26T19:50:35.144169+0000","last_became_peered":"2024-04-26T19:50:35.144169+0000","last_unstale":"2024-04-26T19:50:37.130140+0000","last_undegraded":"2024-04-26T19:50:37.130140+0000","last_fullsized":"2024-04-26T19:50:37.130140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300094+0000","last_change":"2024-04-26T19:50:31.105626+0000","last_active":"2024-04-26T19:50:38.300094+0000","last_peered":"2024-04-26T19:50:38.300094+0000","last_clean":"2024-04-26T19:50:38.300094+0000","last_became_active":"2024-04-26T19:50:31.105369+0000","last_became_peered":"2024-04-26T19:50:31.105369+0000","last_unstale":"2024-04-26T19:50:38.300094+0000","last_undegraded":"2024-04-26T19:50:38.300094+0000","last_fullsized":"2024-04-26T19:50:38.300094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654017+0000","last_change":"2024-04-26T19:50:30.104092+0000","last_active":"2024-04-26T19:50:38.654017+0000","last_peered":"2024-04-26T19:50:38.654017+0000","last_clean":"2024-04-26T19:50:38.654017+0000","last_became_active":"2024-04-26T19:50:30.103848+0000","last_became_peered":"2024-04-26T19:50:30.103848+0000","last_unstale":"2024-04-26T19:50:38.654017+0000","last_undegraded":"2024-04-26T19:50:38.654017+0000","last_fullsized":"2024-04-26T19:50:38.654017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129689+0000","last_change":"2024-04-26T19:50:35.624499+0000","last_active":"2024-04-26T19:50:37.129689+0000","last_peered":"2024-04-26T19:50:37.129689+0000","last_clean":"2024-04-26T19:50:37.129689+0000","last_became_active":"2024-04-26T19:50:35.624026+0000","last_became_peered":"2024-04-26T19:50:35.624026+0000","last_unstale":"2024-04-26T19:50:37.129689+0000","last_undegraded":"2024-04-26T19:50:37.129689+0000","last_fullsized":"2024-04-26T19:50:37.129689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:46.591736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131049+0000","last_change":"2024-04-26T19:50:31.116347+0000","last_active":"2024-04-26T19:50:37.131049+0000","last_peered":"2024-04-26T19:50:37.131049+0000","last_clean":"2024-04-26T19:50:37.131049+0000","last_became_active":"2024-04-26T19:50:31.116037+0000","last_became_peered":"2024-04-26T19:50:31.116037+0000","last_unstale":"2024-04-26T19:50:37.131049+0000","last_undegraded":"2024-04-26T19:50:37.131049+0000","last_fullsized":"2024-04-26T19:50:37.131049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997937+0000","last_change":"2024-04-26T19:50:30.105663+0000","last_active":"2024-04-26T19:50:37.997937+0000","last_peered":"2024-04-26T19:50:37.997937+0000","last_clean":"2024-04-26T19:50:37.997937+0000","last_became_active":"2024-04-26T19:50:30.105387+0000","last_became_peered":"2024-04-26T19:50:30.105387+0000","last_unstale":"2024-04-26T19:50:37.997937+0000","last_undegraded":"2024-04-26T19:50:37.997937+0000","last_fullsized":"2024-04-26T19:50:37.997937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130198+0000","last_change":"2024-04-26T19:50:35.625444+0000","last_active":"2024-04-26T19:50:37.130198+0000","last_peered":"2024-04-26T19:50:37.130198+0000","last_clean":"2024-04-26T19:50:37.130198+0000","last_became_active":"2024-04-26T19:50:35.625054+0000","last_became_peered":"2024-04-26T19:50:35.625054+0000","last_unstale":"2024-04-26T19:50:37.130198+0000","last_undegraded":"2024-04-26T19:50:37.130198+0000","last_fullsized":"2024-04-26T19:50:37.130198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:07.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997757+0000","last_change":"2024-04-26T19:50:31.109145+0000","last_active":"2024-04-26T19:50:37.997757+0000","last_peered":"2024-04-26T19:50:37.997757+0000","last_clean":"2024-04-26T19:50:37.997757+0000","last_became_active":"2024-04-26T19:50:31.108812+0000","last_became_peered":"2024-04-26T19:50:31.108812+0000","last_unstale":"2024-04-26T19:50:37.997757+0000","last_undegraded":"2024-04-26T19:50:37.997757+0000","last_fullsized":"2024-04-26T19:50:37.997757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653987+0000","last_change":"2024-04-26T19:50:30.112025+0000","last_active":"2024-04-26T19:50:38.653987+0000","last_peered":"2024-04-26T19:50:38.653987+0000","last_clean":"2024-04-26T19:50:38.653987+0000","last_became_active":"2024-04-26T19:50:30.111777+0000","last_became_peered":"2024-04-26T19:50:30.111777+0000","last_unstale":"2024-04-26T19:50:38.653987+0000","last_undegraded":"2024-04-26T19:50:38.653987+0000","last_fullsized":"2024-04-26T19:50:38.653987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:41.838039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299918+0000","last_change":"2024-04-26T19:50:35.623970+0000","last_active":"2024-04-26T19:50:38.299918+0000","last_peered":"2024-04-26T19:50:38.299918+0000","last_clean":"2024-04-26T19:50:38.299918+0000","last_became_active":"2024-04-26T19:50:35.623472+0000","last_became_peered":"2024-04-26T19:50:35.623472+0000","last_unstale":"2024-04-26T19:50:38.299918+0000","last_undegraded":"2024-04-26T19:50:38.299918+0000","last_fullsized":"2024-04-26T19:50:38.299918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654678+0000","last_change":"2024-04-26T19:50:31.120338+0000","last_active":"2024-04-26T19:50:38.654678+0000","last_peered":"2024-04-26T19:50:38.654678+0000","last_clean":"2024-04-26T19:50:38.654678+0000","last_became_active":"2024-04-26T19:50:31.120074+0000","last_became_peered":"2024-04-26T19:50:31.120074+0000","last_unstale":"2024-04-26T19:50:38.654678+0000","last_undegraded":"2024-04-26T19:50:38.654678+0000","last_fullsized":"2024-04-26T19:50:38.654678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997828+0000","last_change":"2024-04-26T19:50:30.106832+0000","last_active":"2024-04-26T19:50:37.997828+0000","last_peered":"2024-04-26T19:50:37.997828+0000","last_clean":"2024-04-26T19:50:37.997828+0000","last_became_active":"2024-04-26T19:50:30.106553+0000","last_became_peered":"2024-04-26T19:50:30.106553+0000","last_unstale":"2024-04-26T19:50:37.997828+0000","last_undegraded":"2024-04-26T19:50:37.997828+0000","last_fullsized":"2024-04-26T19:50:37.997828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997421+0000","last_change":"2024-04-26T19:50:35.149679+0000","last_active":"2024-04-26T19:50:37.997421+0000","last_peered":"2024-04-26T19:50:37.997421+0000","last_clean":"2024-04-26T19:50:37.997421+0000","last_became_active":"2024-04-26T19:50:35.149304+0000","last_became_peered":"2024-04-26T19:50:35.149304+0000","last_unstale":"2024-04-26T19:50:37.997421+0000","last_undegraded":"2024-04-26T19:50:37.997421+0000","last_fullsized":"2024-04-26T19:50:37.997421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130385+0000","last_change":"2024-04-26T19:50:31.105502+0000","last_active":"2024-04-26T19:50:37.130385+0000","last_peered":"2024-04-26T19:50:37.130385+0000","last_clean":"2024-04-26T19:50:37.130385+0000","last_became_active":"2024-04-26T19:50:31.105047+0000","last_became_peered":"2024-04-26T19:50:31.105047+0000","last_unstale":"2024-04-26T19:50:37.130385+0000","last_undegraded":"2024-04-26T19:50:37.130385+0000","last_fullsized":"2024-04-26T19:50:37.130385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:20.874838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653576+0000","last_change":"2024-04-26T19:50:30.110096+0000","last_active":"2024-04-26T19:50:38.653576+0000","last_peered":"2024-04-26T19:50:38.653576+0000","last_clean":"2024-04-26T19:50:38.653576+0000","last_became_active":"2024-04-26T19:50:30.109131+0000","last_became_peered":"2024-04-26T19:50:30.109131+0000","last_unstale":"2024-04-26T19:50:38.653576+0000","last_undegraded":"2024-04-26T19:50:38.653576+0000","last_fullsized":"2024-04-26T19:50:38.653576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997740+0000","last_change":"2024-04-26T19:50:35.148072+0000","last_active":"2024-04-26T19:50:37.997740+0000","last_peered":"2024-04-26T19:50:37.997740+0000","last_clean":"2024-04-26T19:50:37.997740+0000","last_became_active":"2024-04-26T19:50:35.147697+0000","last_became_peered":"2024-04-26T19:50:35.147697+0000","last_unstale":"2024-04-26T19:50:37.997740+0000","last_undegraded":"2024-04-26T19:50:37.997740+0000","last_fullsized":"2024-04-26T19:50:37.997740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997366+0000","last_change":"2024-04-26T19:50:31.108339+0000","last_active":"2024-04-26T19:50:37.997366+0000","last_peered":"2024-04-26T19:50:37.997366+0000","last_clean":"2024-04-26T19:50:37.997366+0000","last_became_active":"2024-04-26T19:50:31.108056+0000","last_became_peered":"2024-04-26T19:50:31.108056+0000","last_unstale":"2024-04-26T19:50:37.997366+0000","last_undegraded":"2024-04-26T19:50:37.997366+0000","last_fullsized":"2024-04-26T19:50:37.997366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536841+0000","last_change":"2024-04-26T19:50:30.111350+0000","last_active":"2024-04-26T19:50:38.536841+0000","last_peered":"2024-04-26T19:50:38.536841+0000","last_clean":"2024-04-26T19:50:38.536841+0000","last_became_active":"2024-04-26T19:50:30.111070+0000","last_became_peered":"2024-04-26T19:50:30.111070+0000","last_unstale":"2024-04-26T19:50:38.536841+0000","last_undegraded":"2024-04-26T19:50:38.536841+0000","last_fullsized":"2024-04-26T19:50:38.536841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.998002+0000","last_change":"2024-04-26T19:50:35.146694+0000","last_active":"2024-04-26T19:50:37.998002+0000","last_peered":"2024-04-26T19:50:37.998002+0000","last_clean":"2024-04-26T19:50:37.998002+0000","last_became_active":"2024-04-26T19:50:35.145805+0000","last_became_peered":"2024-04-26T19:50:35.145805+0000","last_unstale":"2024-04-26T19:50:37.998002+0000","last_undegraded":"2024-04-26T19:50:37.998002+0000","last_fullsized":"2024-04-26T19:50:37.998002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:01.350601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299548+0000","last_change":"2024-04-26T19:50:31.103279+0000","last_active":"2024-04-26T19:50:38.299548+0000","last_peered":"2024-04-26T19:50:38.299548+0000","last_clean":"2024-04-26T19:50:38.299548+0000","last_became_active":"2024-04-26T19:50:31.102858+0000","last_became_peered":"2024-04-26T19:50:31.102858+0000","last_unstale":"2024-04-26T19:50:38.299548+0000","last_undegraded":"2024-04-26T19:50:38.299548+0000","last_fullsized":"2024-04-26T19:50:38.299548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:56.497144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653771+0000","last_change":"2024-04-26T19:50:30.109372+0000","last_active":"2024-04-26T19:50:38.653771+0000","last_peered":"2024-04-26T19:50:38.653771+0000","last_clean":"2024-04-26T19:50:38.653771+0000","last_became_active":"2024-04-26T19:50:30.109087+0000","last_became_peered":"2024-04-26T19:50:30.109087+0000","last_unstale":"2024-04-26T19:50:38.653771+0000","last_undegraded":"2024-04-26T19:50:38.653771+0000","last_fullsized":"2024-04-26T19:50:38.653771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:13.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300021+0000","last_change":"2024-04-26T19:50:35.623867+0000","last_active":"2024-04-26T19:50:38.300021+0000","last_peered":"2024-04-26T19:50:38.300021+0000","last_clean":"2024-04-26T19:50:38.300021+0000","last_became_active":"2024-04-26T19:50:35.623416+0000","last_became_peered":"2024-04-26T19:50:35.623416+0000","last_unstale":"2024-04-26T19:50:38.300021+0000","last_undegraded":"2024-04-26T19:50:38.300021+0000","last_fullsized":"2024-04-26T19:50:38.300021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:02.874670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130394+0000","last_change":"2024-04-26T19:50:31.113014+0000","last_active":"2024-04-26T19:50:37.130394+0000","last_peered":"2024-04-26T19:50:37.130394+0000","last_clean":"2024-04-26T19:50:37.130394+0000","last_became_active":"2024-04-26T19:50:31.112753+0000","last_became_peered":"2024-04-26T19:50:31.112753+0000","last_unstale":"2024-04-26T19:50:37.130394+0000","last_undegraded":"2024-04-26T19:50:37.130394+0000","last_fullsized":"2024-04-26T19:50:37.130394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653084+0000","last_change":"2024-04-26T19:50:30.110312+0000","last_active":"2024-04-26T19:50:38.653084+0000","last_peered":"2024-04-26T19:50:38.653084+0000","last_clean":"2024-04-26T19:50:38.653084+0000","last_became_active":"2024-04-26T19:50:30.109147+0000","last_became_peered":"2024-04-26T19:50:30.109147+0000","last_unstale":"2024-04-26T19:50:38.653084+0000","last_undegraded":"2024-04-26T19:50:38.653084+0000","last_fullsized":"2024-04-26T19:50:38.653084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654171+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:50:38.654171+0000","last_peered":"2024-04-26T19:50:38.654171+0000","last_clean":"2024-04-26T19:50:38.654171+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:50:38.654171+0000","last_undegraded":"2024-04-26T19:50:38.654171+0000","last_fullsized":"2024-04-26T19:50:38.654171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653046+0000","last_change":"2024-04-26T19:50:35.627345+0000","last_active":"2024-04-26T19:50:38.653046+0000","last_peered":"2024-04-26T19:50:38.653046+0000","last_clean":"2024-04-26T19:50:38.653046+0000","last_became_active":"2024-04-26T19:50:35.626795+0000","last_became_peered":"2024-04-26T19:50:35.626795+0000","last_unstale":"2024-04-26T19:50:38.653046+0000","last_undegraded":"2024-04-26T19:50:38.653046+0000","last_fullsized":"2024-04-26T19:50:38.653046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:14.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653270+0000","last_change":"2024-04-26T19:50:31.112950+0000","last_active":"2024-04-26T19:50:38.653270+0000","last_peered":"2024-04-26T19:50:38.653270+0000","last_clean":"2024-04-26T19:50:38.653270+0000","last_became_active":"2024-04-26T19:50:31.112674+0000","last_became_peered":"2024-04-26T19:50:31.112674+0000","last_unstale":"2024-04-26T19:50:38.653270+0000","last_undegraded":"2024-04-26T19:50:38.653270+0000","last_fullsized":"2024-04-26T19:50:38.653270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537253+0000","last_change":"2024-04-26T19:50:30.114819+0000","last_active":"2024-04-26T19:50:38.537253+0000","last_peered":"2024-04-26T19:50:38.537253+0000","last_clean":"2024-04-26T19:50:38.537253+0000","last_became_active":"2024-04-26T19:50:30.114566+0000","last_became_peered":"2024-04-26T19:50:30.114566+0000","last_unstale":"2024-04-26T19:50:38.537253+0000","last_undegraded":"2024-04-26T19:50:38.537253+0000","last_fullsized":"2024-04-26T19:50:38.537253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654387+0000","last_change":"2024-04-26T19:50:35.626488+0000","last_active":"2024-04-26T19:50:38.654387+0000","last_peered":"2024-04-26T19:50:38.654387+0000","last_clean":"2024-04-26T19:50:38.654387+0000","last_became_active":"2024-04-26T19:50:35.625951+0000","last_became_peered":"2024-04-26T19:50:35.625951+0000","last_unstale":"2024-04-26T19:50:38.654387+0000","last_undegraded":"2024-04-26T19:50:38.654387+0000","last_fullsized":"2024-04-26T19:50:38.654387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:46.591736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652684+0000","last_change":"2024-04-26T19:50:31.119380+0000","last_active":"2024-04-26T19:50:38.652684+0000","last_peered":"2024-04-26T19:50:38.652684+0000","last_clean":"2024-04-26T19:50:38.652684+0000","last_became_active":"2024-04-26T19:50:31.118011+0000","last_became_peered":"2024-04-26T19:50:31.118011+0000","last_unstale":"2024-04-26T19:50:38.652684+0000","last_undegraded":"2024-04-26T19:50:38.652684+0000","last_fullsized":"2024-04-26T19:50:38.652684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:13.577254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997787+0000","last_change":"2024-04-26T19:50:30.101571+0000","last_active":"2024-04-26T19:50:37.997787+0000","last_peered":"2024-04-26T19:50:37.997787+0000","last_clean":"2024-04-26T19:50:37.997787+0000","last_became_active":"2024-04-26T19:50:30.101257+0000","last_became_peered":"2024-04-26T19:50:30.101257+0000","last_unstale":"2024-04-26T19:50:37.997787+0000","last_undegraded":"2024-04-26T19:50:37.997787+0000","last_fullsized":"2024-04-26T19:50:37.997787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:13.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299968+0000","last_change":"2024-04-26T19:50:35.146304+0000","last_active":"2024-04-26T19:50:38.299968+0000","last_peered":"2024-04-26T19:50:38.299968+0000","last_clean":"2024-04-26T19:50:38.299968+0000","last_became_active":"2024-04-26T19:50:35.145918+0000","last_became_peered":"2024-04-26T19:50:35.145918+0000","last_unstale":"2024-04-26T19:50:38.299968+0000","last_undegraded":"2024-04-26T19:50:38.299968+0000","last_fullsized":"2024-04-26T19:50:38.299968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653338+0000","last_change":"2024-04-26T19:50:31.115622+0000","last_active":"2024-04-26T19:50:38.653338+0000","last_peered":"2024-04-26T19:50:38.653338+0000","last_clean":"2024-04-26T19:50:38.653338+0000","last_became_active":"2024-04-26T19:50:31.115358+0000","last_became_peered":"2024-04-26T19:50:31.115358+0000","last_unstale":"2024-04-26T19:50:38.653338+0000","last_undegraded":"2024-04-26T19:50:38.653338+0000","last_fullsized":"2024-04-26T19:50:38.653338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:31.186700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536869+0000","last_change":"2024-04-26T19:50:30.112087+0000","last_active":"2024-04-26T19:50:38.536869+0000","last_peered":"2024-04-26T19:50:38.536869+0000","last_clean":"2024-04-26T19:50:38.536869+0000","last_became_active":"2024-04-26T19:50:30.111773+0000","last_became_peered":"2024-04-26T19:50:30.111773+0000","last_unstale":"2024-04-26T19:50:38.536869+0000","last_undegraded":"2024-04-26T19:50:38.536869+0000","last_fullsized":"2024-04-26T19:50:38.536869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652160+0000","last_change":"2024-04-26T19:50:35.624170+0000","last_active":"2024-04-26T19:50:38.652160+0000","last_peered":"2024-04-26T19:50:38.652160+0000","last_clean":"2024-04-26T19:50:38.652160+0000","last_became_active":"2024-04-26T19:50:35.623758+0000","last_became_peered":"2024-04-26T19:50:35.623758+0000","last_unstale":"2024-04-26T19:50:38.652160+0000","last_undegraded":"2024-04-26T19:50:38.652160+0000","last_fullsized":"2024-04-26T19:50:38.652160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:01.350601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653614+0000","last_change":"2024-04-26T19:50:31.115100+0000","last_active":"2024-04-26T19:50:38.653614+0000","last_peered":"2024-04-26T19:50:38.653614+0000","last_clean":"2024-04-26T19:50:38.653614+0000","last_became_active":"2024-04-26T19:50:31.114558+0000","last_became_peered":"2024-04-26T19:50:31.114558+0000","last_unstale":"2024-04-26T19:50:38.653614+0000","last_undegraded":"2024-04-26T19:50:38.653614+0000","last_fullsized":"2024-04-26T19:50:38.653614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:42.558104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130407+0000","last_change":"2024-04-26T19:50:30.100580+0000","last_active":"2024-04-26T19:50:37.130407+0000","last_peered":"2024-04-26T19:50:37.130407+0000","last_clean":"2024-04-26T19:50:37.130407+0000","last_became_active":"2024-04-26T19:50:30.099562+0000","last_became_peered":"2024-04-26T19:50:30.099562+0000","last_unstale":"2024-04-26T19:50:37.130407+0000","last_undegraded":"2024-04-26T19:50:37.130407+0000","last_fullsized":"2024-04-26T19:50:37.130407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:02.207190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129529+0000","last_change":"2024-04-26T19:50:35.143446+0000","last_active":"2024-04-26T19:50:37.129529+0000","last_peered":"2024-04-26T19:50:37.129529+0000","last_clean":"2024-04-26T19:50:37.129529+0000","last_became_active":"2024-04-26T19:50:35.143028+0000","last_became_peered":"2024-04-26T19:50:35.143028+0000","last_unstale":"2024-04-26T19:50:37.129529+0000","last_undegraded":"2024-04-26T19:50:37.129529+0000","last_fullsized":"2024-04-26T19:50:37.129529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:43.974896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653088+0000","last_change":"2024-04-26T19:50:31.116546+0000","last_active":"2024-04-26T19:50:38.653088+0000","last_peered":"2024-04-26T19:50:38.653088+0000","last_clean":"2024-04-26T19:50:38.653088+0000","last_became_active":"2024-04-26T19:50:31.115261+0000","last_became_peered":"2024-04-26T19:50:31.115261+0000","last_unstale":"2024-04-26T19:50:38.653088+0000","last_undegraded":"2024-04-26T19:50:38.653088+0000","last_fullsized":"2024-04-26T19:50:38.653088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.212877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536837+0000","last_change":"2024-04-26T19:50:30.099768+0000","last_active":"2024-04-26T19:50:38.536837+0000","last_peered":"2024-04-26T19:50:38.536837+0000","last_clean":"2024-04-26T19:50:38.536837+0000","last_became_active":"2024-04-26T19:50:30.099471+0000","last_became_peered":"2024-04-26T19:50:30.099471+0000","last_unstale":"2024-04-26T19:50:38.536837+0000","last_undegraded":"2024-04-26T19:50:38.536837+0000","last_fullsized":"2024-04-26T19:50:38.536837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299923+0000","last_change":"2024-04-26T19:50:35.148077+0000","last_active":"2024-04-26T19:50:38.299923+0000","last_peered":"2024-04-26T19:50:38.299923+0000","last_clean":"2024-04-26T19:50:38.299923+0000","last_became_active":"2024-04-26T19:50:35.147552+0000","last_became_peered":"2024-04-26T19:50:35.147552+0000","last_unstale":"2024-04-26T19:50:38.299923+0000","last_undegraded":"2024-04-26T19:50:38.299923+0000","last_fullsized":"2024-04-26T19:50:38.299923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:11.052818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652600+0000","last_change":"2024-04-26T19:50:31.105398+0000","last_active":"2024-04-26T19:50:38.652600+0000","last_peered":"2024-04-26T19:50:38.652600+0000","last_clean":"2024-04-26T19:50:38.652600+0000","last_became_active":"2024-04-26T19:50:31.105140+0000","last_became_peered":"2024-04-26T19:50:31.105140+0000","last_unstale":"2024-04-26T19:50:38.652600+0000","last_undegraded":"2024-04-26T19:50:38.652600+0000","last_fullsized":"2024-04-26T19:50:38.652600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.651966+0000","last_change":"2024-04-26T19:50:30.111019+0000","last_active":"2024-04-26T19:50:38.651966+0000","last_peered":"2024-04-26T19:50:38.651966+0000","last_clean":"2024-04-26T19:50:38.651966+0000","last_became_active":"2024-04-26T19:50:30.110590+0000","last_became_peered":"2024-04-26T19:50:30.110590+0000","last_unstale":"2024-04-26T19:50:38.651966+0000","last_undegraded":"2024-04-26T19:50:38.651966+0000","last_fullsized":"2024-04-26T19:50:38.651966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131001+0000","last_change":"2024-04-26T19:50:35.625721+0000","last_active":"2024-04-26T19:50:37.131001+0000","last_peered":"2024-04-26T19:50:37.131001+0000","last_clean":"2024-04-26T19:50:37.131001+0000","last_became_active":"2024-04-26T19:50:35.625294+0000","last_became_peered":"2024-04-26T19:50:35.625294+0000","last_unstale":"2024-04-26T19:50:37.131001+0000","last_undegraded":"2024-04-26T19:50:37.131001+0000","last_fullsized":"2024-04-26T19:50:37.131001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:18.158960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653765+0000","last_change":"2024-04-26T19:50:31.113984+0000","last_active":"2024-04-26T19:50:38.653765+0000","last_peered":"2024-04-26T19:50:38.653765+0000","last_clean":"2024-04-26T19:50:38.653765+0000","last_became_active":"2024-04-26T19:50:31.113723+0000","last_became_peered":"2024-04-26T19:50:31.113723+0000","last_unstale":"2024-04-26T19:50:38.653765+0000","last_undegraded":"2024-04-26T19:50:38.653765+0000","last_fullsized":"2024-04-26T19:50:38.653765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653698+0000","last_change":"2024-04-26T19:50:30.110687+0000","last_active":"2024-04-26T19:50:38.653698+0000","last_peered":"2024-04-26T19:50:38.653698+0000","last_clean":"2024-04-26T19:50:38.653698+0000","last_became_active":"2024-04-26T19:50:30.110423+0000","last_became_peered":"2024-04-26T19:50:30.110423+0000","last_unstale":"2024-04-26T19:50:38.653698+0000","last_undegraded":"2024-04-26T19:50:38.653698+0000","last_fullsized":"2024-04-26T19:50:38.653698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130741+0000","last_change":"2024-04-26T19:50:35.623855+0000","last_active":"2024-04-26T19:50:37.130741+0000","last_peered":"2024-04-26T19:50:37.130741+0000","last_clean":"2024-04-26T19:50:37.130741+0000","last_became_active":"2024-04-26T19:50:35.623453+0000","last_became_peered":"2024-04-26T19:50:35.623453+0000","last_unstale":"2024-04-26T19:50:37.130741+0000","last_undegraded":"2024-04-26T19:50:37.130741+0000","last_fullsized":"2024-04-26T19:50:37.130741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:21.199810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299726+0000","last_change":"2024-04-26T19:50:31.114891+0000","last_active":"2024-04-26T19:50:38.299726+0000","last_peered":"2024-04-26T19:50:38.299726+0000","last_clean":"2024-04-26T19:50:38.299726+0000","last_became_active":"2024-04-26T19:50:31.114338+0000","last_became_peered":"2024-04-26T19:50:31.114338+0000","last_unstale":"2024-04-26T19:50:38.299726+0000","last_undegraded":"2024-04-26T19:50:38.299726+0000","last_fullsized":"2024-04-26T19:50:38.299726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299301+0000","last_change":"2024-04-26T19:50:30.108408+0000","last_active":"2024-04-26T19:50:38.299301+0000","last_peered":"2024-04-26T19:50:38.299301+0000","last_clean":"2024-04-26T19:50:38.299301+0000","last_became_active":"2024-04-26T19:50:30.107261+0000","last_became_peered":"2024-04-26T19:50:30.107261+0000","last_unstale":"2024-04-26T19:50:38.299301+0000","last_undegraded":"2024-04-26T19:50:38.299301+0000","last_fullsized":"2024-04-26T19:50:38.299301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:21.153340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654534+0000","last_change":"2024-04-26T19:50:35.624403+0000","last_active":"2024-04-26T19:50:38.654534+0000","last_peered":"2024-04-26T19:50:38.654534+0000","last_clean":"2024-04-26T19:50:38.654534+0000","last_became_active":"2024-04-26T19:50:35.623811+0000","last_became_peered":"2024-04-26T19:50:35.623811+0000","last_unstale":"2024-04-26T19:50:38.654534+0000","last_undegraded":"2024-04-26T19:50:38.654534+0000","last_fullsized":"2024-04-26T19:50:38.654534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.747785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653171+0000","last_change":"2024-04-26T19:50:31.115452+0000","last_active":"2024-04-26T19:50:38.653171+0000","last_peered":"2024-04-26T19:50:38.653171+0000","last_clean":"2024-04-26T19:50:38.653171+0000","last_became_active":"2024-04-26T19:50:31.114780+0000","last_became_peered":"2024-04-26T19:50:31.114780+0000","last_unstale":"2024-04-26T19:50:38.653171+0000","last_undegraded":"2024-04-26T19:50:38.653171+0000","last_fullsized":"2024-04-26T19:50:38.653171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:40.502106+0000","last_change":"2024-04-26T19:50:30.110540+0000","last_active":"2024-04-26T19:50:40.502106+0000","last_peered":"2024-04-26T19:50:40.502106+0000","last_clean":"2024-04-26T19:50:40.502106+0000","last_became_active":"2024-04-26T19:50:30.110241+0000","last_became_peered":"2024-04-26T19:50:30.110241+0000","last_unstale":"2024-04-26T19:50:40.502106+0000","last_undegraded":"2024-04-26T19:50:40.502106+0000","last_fullsized":"2024-04-26T19:50:40.502106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997615+0000","last_change":"2024-04-26T19:50:35.149139+0000","last_active":"2024-04-26T19:50:37.997615+0000","last_peered":"2024-04-26T19:50:37.997615+0000","last_clean":"2024-04-26T19:50:37.997615+0000","last_became_active":"2024-04-26T19:50:35.148663+0000","last_became_peered":"2024-04-26T19:50:35.148663+0000","last_unstale":"2024-04-26T19:50:37.997615+0000","last_undegraded":"2024-04-26T19:50:37.997615+0000","last_fullsized":"2024-04-26T19:50:37.997615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130224+0000","last_change":"2024-04-26T19:50:31.112921+0000","last_active":"2024-04-26T19:50:37.130224+0000","last_peered":"2024-04-26T19:50:37.130224+0000","last_clean":"2024-04-26T19:50:37.130224+0000","last_became_active":"2024-04-26T19:50:31.112651+0000","last_became_peered":"2024-04-26T19:50:31.112651+0000","last_unstale":"2024-04-26T19:50:37.130224+0000","last_undegraded":"2024-04-26T19:50:37.130224+0000","last_fullsized":"2024-04-26T19:50:37.130224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:13.149763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997777+0000","last_change":"2024-04-26T19:50:30.110464+0000","last_active":"2024-04-26T19:50:37.997777+0000","last_peered":"2024-04-26T19:50:37.997777+0000","last_clean":"2024-04-26T19:50:37.997777+0000","last_became_active":"2024-04-26T19:50:30.110101+0000","last_became_peered":"2024-04-26T19:50:30.110101+0000","last_unstale":"2024-04-26T19:50:37.997777+0000","last_undegraded":"2024-04-26T19:50:37.997777+0000","last_fullsized":"2024-04-26T19:50:37.997777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:12.144175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130067+0000","last_change":"2024-04-26T19:50:35.143854+0000","last_active":"2024-04-26T19:50:37.130067+0000","last_peered":"2024-04-26T19:50:37.130067+0000","last_clean":"2024-04-26T19:50:37.130067+0000","last_became_active":"2024-04-26T19:50:35.143264+0000","last_became_peered":"2024-04-26T19:50:35.143264+0000","last_unstale":"2024-04-26T19:50:37.130067+0000","last_undegraded":"2024-04-26T19:50:37.130067+0000","last_fullsized":"2024-04-26T19:50:37.130067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300209+0000","last_change":"2024-04-26T19:50:31.115638+0000","last_active":"2024-04-26T19:50:38.300209+0000","last_peered":"2024-04-26T19:50:38.300209+0000","last_clean":"2024-04-26T19:50:38.300209+0000","last_became_active":"2024-04-26T19:50:31.113889+0000","last_became_peered":"2024-04-26T19:50:31.113889+0000","last_unstale":"2024-04-26T19:50:38.300209+0000","last_undegraded":"2024-04-26T19:50:38.300209+0000","last_fullsized":"2024-04-26T19:50:38.300209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652958+0000","last_change":"2024-04-26T19:50:30.109848+0000","last_active":"2024-04-26T19:50:38.652958+0000","last_peered":"2024-04-26T19:50:38.652958+0000","last_clean":"2024-04-26T19:50:38.652958+0000","last_became_active":"2024-04-26T19:50:30.109407+0000","last_became_peered":"2024-04-26T19:50:30.109407+0000","last_unstale":"2024-04-26T19:50:38.652958+0000","last_undegraded":"2024-04-26T19:50:38.652958+0000","last_fullsized":"2024-04-26T19:50:38.652958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130343+0000","last_change":"2024-04-26T19:50:35.143221+0000","last_active":"2024-04-26T19:50:37.130343+0000","last_peered":"2024-04-26T19:50:37.130343+0000","last_clean":"2024-04-26T19:50:37.130343+0000","last_became_active":"2024-04-26T19:50:35.142868+0000","last_became_peered":"2024-04-26T19:50:35.142868+0000","last_unstale":"2024-04-26T19:50:37.130343+0000","last_undegraded":"2024-04-26T19:50:37.130343+0000","last_fullsized":"2024-04-26T19:50:37.130343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:16.097115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652135+0000","last_change":"2024-04-26T19:50:31.115770+0000","last_active":"2024-04-26T19:50:38.652135+0000","last_peered":"2024-04-26T19:50:38.652135+0000","last_clean":"2024-04-26T19:50:38.652135+0000","last_became_active":"2024-04-26T19:50:31.115462+0000","last_became_peered":"2024-04-26T19:50:31.115462+0000","last_unstale":"2024-04-26T19:50:38.652135+0000","last_undegraded":"2024-04-26T19:50:38.652135+0000","last_fullsized":"2024-04-26T19:50:38.652135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.212877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653880+0000","last_change":"2024-04-26T19:50:30.110213+0000","last_active":"2024-04-26T19:50:38.653880+0000","last_peered":"2024-04-26T19:50:38.653880+0000","last_clean":"2024-04-26T19:50:38.653880+0000","last_became_active":"2024-04-26T19:50:30.109923+0000","last_became_peered":"2024-04-26T19:50:30.109923+0000","last_unstale":"2024-04-26T19:50:38.653880+0000","last_undegraded":"2024-04-26T19:50:38.653880+0000","last_fullsized":"2024-04-26T19:50:38.653880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130324+0000","last_change":"2024-04-26T19:50:35.624637+0000","last_active":"2024-04-26T19:50:37.130324+0000","last_peered":"2024-04-26T19:50:37.130324+0000","last_clean":"2024-04-26T19:50:37.130324+0000","last_became_active":"2024-04-26T19:50:35.624195+0000","last_became_peered":"2024-04-26T19:50:35.624195+0000","last_unstale":"2024-04-26T19:50:37.130324+0000","last_undegraded":"2024-04-26T19:50:37.130324+0000","last_fullsized":"2024-04-26T19:50:37.130324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:16.049053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299925+0000","last_change":"2024-04-26T19:50:35.626111+0000","last_active":"2024-04-26T19:50:38.299925+0000","last_peered":"2024-04-26T19:50:38.299925+0000","last_clean":"2024-04-26T19:50:38.299925+0000","last_became_active":"2024-04-26T19:50:35.625678+0000","last_became_peered":"2024-04-26T19:50:35.625678+0000","last_unstale":"2024-04-26T19:50:38.299925+0000","last_undegraded":"2024-04-26T19:50:38.299925+0000","last_fullsized":"2024-04-26T19:50:38.299925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997291+0000","last_change":"2024-04-26T19:50:31.108163+0000","last_active":"2024-04-26T19:50:37.997291+0000","last_peered":"2024-04-26T19:50:37.997291+0000","last_clean":"2024-04-26T19:50:37.997291+0000","last_became_active":"2024-04-26T19:50:31.107840+0000","last_became_peered":"2024-04-26T19:50:31.107840+0000","last_unstale":"2024-04-26T19:50:37.997291+0000","last_undegraded":"2024-04-26T19:50:37.997291+0000","last_fullsized":"2024-04-26T19:50:37.997291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.651925+0000","last_change":"2024-04-26T19:50:30.111205+0000","last_active":"2024-04-26T19:50:38.651925+0000","last_peered":"2024-04-26T19:50:38.651925+0000","last_clean":"2024-04-26T19:50:38.651925+0000","last_became_active":"2024-04-26T19:50:30.110491+0000","last_became_peered":"2024-04-26T19:50:30.110491+0000","last_unstale":"2024-04-26T19:50:38.651925+0000","last_undegraded":"2024-04-26T19:50:38.651925+0000","last_fullsized":"2024-04-26T19:50:38.651925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653880+0000","last_change":"2024-04-26T19:50:35.144010+0000","last_active":"2024-04-26T19:50:38.653880+0000","last_peered":"2024-04-26T19:50:38.653880+0000","last_clean":"2024-04-26T19:50:38.653880+0000","last_became_active":"2024-04-26T19:50:35.143558+0000","last_became_peered":"2024-04-26T19:50:35.143558+0000","last_unstale":"2024-04-26T19:50:38.653880+0000","last_undegraded":"2024-04-26T19:50:38.653880+0000","last_fullsized":"2024-04-26T19:50:38.653880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:10.325286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130338+0000","last_change":"2024-04-26T19:50:31.106592+0000","last_active":"2024-04-26T19:50:37.130338+0000","last_peered":"2024-04-26T19:50:37.130338+0000","last_clean":"2024-04-26T19:50:37.130338+0000","last_became_active":"2024-04-26T19:50:31.106298+0000","last_became_peered":"2024-04-26T19:50:31.106298+0000","last_unstale":"2024-04-26T19:50:37.130338+0000","last_undegraded":"2024-04-26T19:50:37.130338+0000","last_fullsized":"2024-04-26T19:50:37.130338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130435+0000","last_change":"2024-04-26T19:50:30.110713+0000","last_active":"2024-04-26T19:50:37.130435+0000","last_peered":"2024-04-26T19:50:37.130435+0000","last_clean":"2024-04-26T19:50:37.130435+0000","last_became_active":"2024-04-26T19:50:30.109274+0000","last_became_peered":"2024-04-26T19:50:30.109274+0000","last_unstale":"2024-04-26T19:50:37.130435+0000","last_undegraded":"2024-04-26T19:50:37.130435+0000","last_fullsized":"2024-04-26T19:50:37.130435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654164+0000","last_change":"2024-04-26T19:50:30.111443+0000","last_active":"2024-04-26T19:50:38.654164+0000","last_peered":"2024-04-26T19:50:38.654164+0000","last_clean":"2024-04-26T19:50:38.654164+0000","last_became_active":"2024-04-26T19:50:30.111159+0000","last_became_peered":"2024-04-26T19:50:30.111159+0000","last_unstale":"2024-04-26T19:50:38.654164+0000","last_undegraded":"2024-04-26T19:50:38.654164+0000","last_fullsized":"2024-04-26T19:50:38.654164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300079+0000","last_change":"2024-04-26T19:50:31.104770+0000","last_active":"2024-04-26T19:50:38.300079+0000","last_peered":"2024-04-26T19:50:38.300079+0000","last_clean":"2024-04-26T19:50:38.300079+0000","last_became_active":"2024-04-26T19:50:31.104100+0000","last_became_peered":"2024-04-26T19:50:31.104100+0000","last_unstale":"2024-04-26T19:50:38.300079+0000","last_undegraded":"2024-04-26T19:50:38.300079+0000","last_fullsized":"2024-04-26T19:50:38.300079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537176+0000","last_change":"2024-04-26T19:50:35.143179+0000","last_active":"2024-04-26T19:50:38.537176+0000","last_peered":"2024-04-26T19:50:38.537176+0000","last_clean":"2024-04-26T19:50:38.537176+0000","last_became_active":"2024-04-26T19:50:35.142761+0000","last_became_peered":"2024-04-26T19:50:35.142761+0000","last_unstale":"2024-04-26T19:50:38.537176+0000","last_undegraded":"2024-04-26T19:50:38.537176+0000","last_fullsized":"2024-04-26T19:50:38.537176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:42.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130613+0000","last_change":"2024-04-26T19:50:30.115508+0000","last_active":"2024-04-26T19:50:37.130613+0000","last_peered":"2024-04-26T19:50:37.130613+0000","last_clean":"2024-04-26T19:50:37.130613+0000","last_became_active":"2024-04-26T19:50:30.115235+0000","last_became_peered":"2024-04-26T19:50:30.115235+0000","last_unstale":"2024-04-26T19:50:37.130613+0000","last_undegraded":"2024-04-26T19:50:37.130613+0000","last_fullsized":"2024-04-26T19:50:37.130613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:31.691383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129914+0000","last_change":"2024-04-26T19:50:31.110879+0000","last_active":"2024-04-26T19:50:37.129914+0000","last_peered":"2024-04-26T19:50:37.129914+0000","last_clean":"2024-04-26T19:50:37.129914+0000","last_became_active":"2024-04-26T19:50:31.110592+0000","last_became_peered":"2024-04-26T19:50:31.110592+0000","last_unstale":"2024-04-26T19:50:37.129914+0000","last_undegraded":"2024-04-26T19:50:37.129914+0000","last_fullsized":"2024-04-26T19:50:37.129914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:54.410618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130887+0000","last_change":"2024-04-26T19:50:35.144257+0000","last_active":"2024-04-26T19:50:37.130887+0000","last_peered":"2024-04-26T19:50:37.130887+0000","last_clean":"2024-04-26T19:50:37.130887+0000","last_became_active":"2024-04-26T19:50:35.143853+0000","last_became_peered":"2024-04-26T19:50:35.143853+0000","last_unstale":"2024-04-26T19:50:37.130887+0000","last_undegraded":"2024-04-26T19:50:37.130887+0000","last_fullsized":"2024-04-26T19:50:37.130887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:05.350339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652235+0000","last_change":"2024-04-26T19:50:31.105355+0000","last_active":"2024-04-26T19:50:38.652235+0000","last_peered":"2024-04-26T19:50:38.652235+0000","last_clean":"2024-04-26T19:50:38.652235+0000","last_became_active":"2024-04-26T19:50:31.105085+0000","last_became_peered":"2024-04-26T19:50:31.105085+0000","last_unstale":"2024-04-26T19:50:38.652235+0000","last_undegraded":"2024-04-26T19:50:38.652235+0000","last_fullsized":"2024-04-26T19:50:38.652235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536962+0000","last_change":"2024-04-26T19:50:31.112415+0000","last_active":"2024-04-26T19:50:38.536962+0000","last_peered":"2024-04-26T19:50:38.536962+0000","last_clean":"2024-04-26T19:50:38.536962+0000","last_became_active":"2024-04-26T19:50:31.112126+0000","last_became_peered":"2024-04-26T19:50:31.112126+0000","last_unstale":"2024-04-26T19:50:38.536962+0000","last_undegraded":"2024-04-26T19:50:38.536962+0000","last_fullsized":"2024-04-26T19:50:38.536962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:13.149763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654244+0000","last_change":"2024-04-26T19:50:31.121580+0000","last_active":"2024-04-26T19:50:38.654244+0000","last_peered":"2024-04-26T19:50:38.654244+0000","last_clean":"2024-04-26T19:50:38.654244+0000","last_became_active":"2024-04-26T19:50:31.121296+0000","last_became_peered":"2024-04-26T19:50:31.121296+0000","last_unstale":"2024-04-26T19:50:38.654244+0000","last_undegraded":"2024-04-26T19:50:38.654244+0000","last_fullsized":"2024-04-26T19:50:38.654244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.291653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300008+0000","last_change":"2024-04-26T19:50:31.106600+0000","last_active":"2024-04-26T19:50:38.300008+0000","last_peered":"2024-04-26T19:50:38.300008+0000","last_clean":"2024-04-26T19:50:38.300008+0000","last_became_active":"2024-04-26T19:50:31.106120+0000","last_became_peered":"2024-04-26T19:50:31.106120+0000","last_unstale":"2024-04-26T19:50:38.300008+0000","last_undegraded":"2024-04-26T19:50:38.300008+0000","last_fullsized":"2024-04-26T19:50:38.300008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130813+0000","last_change":"2024-04-26T19:50:31.113865+0000","last_active":"2024-04-26T19:50:37.130813+0000","last_peered":"2024-04-26T19:50:37.130813+0000","last_clean":"2024-04-26T19:50:37.130813+0000","last_became_active":"2024-04-26T19:50:31.113603+0000","last_became_peered":"2024-04-26T19:50:31.113603+0000","last_unstale":"2024-04-26T19:50:37.130813+0000","last_undegraded":"2024-04-26T19:50:37.130813+0000","last_fullsized":"2024-04-26T19:50:37.130813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997661+0000","last_change":"2024-04-26T19:50:31.113048+0000","last_active":"2024-04-26T19:50:37.997661+0000","last_peered":"2024-04-26T19:50:37.997661+0000","last_clean":"2024-04-26T19:50:37.997661+0000","last_became_active":"2024-04-26T19:50:31.112760+0000","last_became_peered":"2024-04-26T19:50:31.112760+0000","last_unstale":"2024-04-26T19:50:37.997661+0000","last_undegraded":"2024-04-26T19:50:37.997661+0000","last_fullsized":"2024-04-26T19:50:37.997661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653673+0000","last_change":"2024-04-26T19:50:31.116289+0000","last_active":"2024-04-26T19:50:38.653673+0000","last_peered":"2024-04-26T19:50:38.653673+0000","last_clean":"2024-04-26T19:50:38.653673+0000","last_became_active":"2024-04-26T19:50:31.116030+0000","last_became_peered":"2024-04-26T19:50:31.116030+0000","last_unstale":"2024-04-26T19:50:38.653673+0000","last_undegraded":"2024-04-26T19:50:38.653673+0000","last_fullsized":"2024-04-26T19:50:38.653673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130946+0000","last_change":"2024-04-26T19:50:31.106460+0000","last_active":"2024-04-26T19:50:37.130946+0000","last_peered":"2024-04-26T19:50:37.130946+0000","last_clean":"2024-04-26T19:50:37.130946+0000","last_became_active":"2024-04-26T19:50:31.106066+0000","last_became_peered":"2024-04-26T19:50:31.106066+0000","last_unstale":"2024-04-26T19:50:37.130946+0000","last_undegraded":"2024-04-26T19:50:37.130946+0000","last_fullsized":"2024-04-26T19:50:37.130946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652449+0000","last_change":"2024-04-26T19:50:31.121061+0000","last_active":"2024-04-26T19:50:38.652449+0000","last_peered":"2024-04-26T19:50:38.652449+0000","last_clean":"2024-04-26T19:50:38.652449+0000","last_became_active":"2024-04-26T19:50:31.120795+0000","last_became_peered":"2024-04-26T19:50:31.120795+0000","last_unstale":"2024-04-26T19:50:38.652449+0000","last_undegraded":"2024-04-26T19:50:38.652449+0000","last_fullsized":"2024-04-26T19:50:38.652449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130353+0000","last_change":"2024-04-26T19:50:31.108437+0000","last_active":"2024-04-26T19:50:37.130353+0000","last_peered":"2024-04-26T19:50:37.130353+0000","last_clean":"2024-04-26T19:50:37.130353+0000","last_became_active":"2024-04-26T19:50:31.108163+0000","last_became_peered":"2024-04-26T19:50:31.108163+0000","last_unstale":"2024-04-26T19:50:37.130353+0000","last_undegraded":"2024-04-26T19:50:37.130353+0000","last_fullsized":"2024-04-26T19:50:37.130353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654157+0000","last_change":"2024-04-26T19:50:31.117685+0000","last_active":"2024-04-26T19:50:38.654157+0000","last_peered":"2024-04-26T19:50:38.654157+0000","last_clean":"2024-04-26T19:50:38.654157+0000","last_became_active":"2024-04-26T19:50:31.115588+0000","last_became_peered":"2024-04-26T19:50:31.115588+0000","last_unstale":"2024-04-26T19:50:38.654157+0000","last_undegraded":"2024-04-26T19:50:38.654157+0000","last_fullsized":"2024-04-26T19:50:38.654157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:10.399495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299776+0000","last_change":"2024-04-26T19:50:31.115273+0000","last_active":"2024-04-26T19:50:38.299776+0000","last_peered":"2024-04-26T19:50:38.299776+0000","last_clean":"2024-04-26T19:50:38.299776+0000","last_became_active":"2024-04-26T19:50:31.114772+0000","last_became_peered":"2024-04-26T19:50:31.114772+0000","last_unstale":"2024-04-26T19:50:38.299776+0000","last_undegraded":"2024-04-26T19:50:38.299776+0000","last_fullsized":"2024-04-26T19:50:38.299776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653493+0000","last_change":"2024-04-26T19:50:31.110492+0000","last_active":"2024-04-26T19:50:38.653493+0000","last_peered":"2024-04-26T19:50:38.653493+0000","last_clean":"2024-04-26T19:50:38.653493+0000","last_became_active":"2024-04-26T19:50:31.110280+0000","last_became_peered":"2024-04-26T19:50:31.110280+0000","last_unstale":"2024-04-26T19:50:38.653493+0000","last_undegraded":"2024-04-26T19:50:38.653493+0000","last_fullsized":"2024-04-26T19:50:38.653493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653886+0000","last_change":"2024-04-26T19:50:31.114136+0000","last_active":"2024-04-26T19:50:38.653886+0000","last_peered":"2024-04-26T19:50:38.653886+0000","last_clean":"2024-04-26T19:50:38.653886+0000","last_became_active":"2024-04-26T19:50:31.113175+0000","last_became_peered":"2024-04-26T19:50:31.113175+0000","last_unstale":"2024-04-26T19:50:38.653886+0000","last_undegraded":"2024-04-26T19:50:38.653886+0000","last_fullsized":"2024-04-26T19:50:38.653886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:39.941264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537153+0000","last_change":"2024-04-26T19:50:31.108339+0000","last_active":"2024-04-26T19:50:38.537153+0000","last_peered":"2024-04-26T19:50:38.537153+0000","last_clean":"2024-04-26T19:50:38.537153+0000","last_became_active":"2024-04-26T19:50:31.108092+0000","last_became_peered":"2024-04-26T19:50:31.108092+0000","last_unstale":"2024-04-26T19:50:38.537153+0000","last_undegraded":"2024-04-26T19:50:38.537153+0000","last_fullsized":"2024-04-26T19:50:38.537153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:14.980411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653311+0000","last_change":"2024-04-26T19:50:31.116207+0000","last_active":"2024-04-26T19:50:38.653311+0000","last_peered":"2024-04-26T19:50:38.653311+0000","last_clean":"2024-04-26T19:50:38.653311+0000","last_became_active":"2024-04-26T19:50:31.115498+0000","last_became_peered":"2024-04-26T19:50:31.115498+0000","last_unstale":"2024-04-26T19:50:38.653311+0000","last_undegraded":"2024-04-26T19:50:38.653311+0000","last_fullsized":"2024-04-26T19:50:38.653311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:42.558104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652346+0000","last_change":"2024-04-26T19:50:31.117352+0000","last_active":"2024-04-26T19:50:38.652346+0000","last_peered":"2024-04-26T19:50:38.652346+0000","last_clean":"2024-04-26T19:50:38.652346+0000","last_became_active":"2024-04-26T19:50:31.117098+0000","last_became_peered":"2024-04-26T19:50:31.117098+0000","last_unstale":"2024-04-26T19:50:38.652346+0000","last_undegraded":"2024-04-26T19:50:38.652346+0000","last_fullsized":"2024-04-26T19:50:38.652346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997376+0000","last_change":"2024-04-26T19:50:31.113884+0000","last_active":"2024-04-26T19:50:37.997376+0000","last_peered":"2024-04-26T19:50:37.997376+0000","last_clean":"2024-04-26T19:50:37.997376+0000","last_became_active":"2024-04-26T19:50:31.113607+0000","last_became_peered":"2024-04-26T19:50:31.113607+0000","last_unstale":"2024-04-26T19:50:37.997376+0000","last_undegraded":"2024-04-26T19:50:37.997376+0000","last_fullsized":"2024-04-26T19:50:37.997376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299710+0000","last_change":"2024-04-26T19:50:31.110086+0000","last_active":"2024-04-26T19:50:38.299710+0000","last_peered":"2024-04-26T19:50:38.299710+0000","last_clean":"2024-04-26T19:50:38.299710+0000","last_became_active":"2024-04-26T19:50:31.109800+0000","last_became_peered":"2024-04-26T19:50:31.109800+0000","last_unstale":"2024-04-26T19:50:38.299710+0000","last_undegraded":"2024-04-26T19:50:38.299710+0000","last_fullsized":"2024-04-26T19:50:38.299710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:45.295846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130514+0000","last_change":"2024-04-26T19:50:31.109078+0000","last_active":"2024-04-26T19:50:37.130514+0000","last_peered":"2024-04-26T19:50:37.130514+0000","last_clean":"2024-04-26T19:50:37.130514+0000","last_became_active":"2024-04-26T19:50:31.107801+0000","last_became_peered":"2024-04-26T19:50:31.107801+0000","last_unstale":"2024-04-26T19:50:37.130514+0000","last_undegraded":"2024-04-26T19:50:37.130514+0000","last_fullsized":"2024-04-26T19:50:37.130514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130138+0000","last_change":"2024-04-26T19:50:31.103841+0000","last_active":"2024-04-26T19:50:37.130138+0000","last_peered":"2024-04-26T19:50:37.130138+0000","last_clean":"2024-04-26T19:50:37.130138+0000","last_became_active":"2024-04-26T19:50:31.103366+0000","last_became_peered":"2024-04-26T19:50:31.103366+0000","last_unstale":"2024-04-26T19:50:37.130138+0000","last_undegraded":"2024-04-26T19:50:37.130138+0000","last_fullsized":"2024-04-26T19:50:37.130138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:22.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653217+0000","last_change":"2024-04-26T19:50:31.113740+0000","last_active":"2024-04-26T19:50:38.653217+0000","last_peered":"2024-04-26T19:50:38.653217+0000","last_clean":"2024-04-26T19:50:38.653217+0000","last_became_active":"2024-04-26T19:50:31.113431+0000","last_became_peered":"2024-04-26T19:50:31.113431+0000","last_unstale":"2024-04-26T19:50:38.653217+0000","last_undegraded":"2024-04-26T19:50:38.653217+0000","last_fullsized":"2024-04-26T19:50:38.653217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653594+0000","last_change":"2024-04-26T19:50:31.113198+0000","last_active":"2024-04-26T19:50:38.653594+0000","last_peered":"2024-04-26T19:50:38.653594+0000","last_clean":"2024-04-26T19:50:38.653594+0000","last_became_active":"2024-04-26T19:50:31.112582+0000","last_became_peered":"2024-04-26T19:50:31.112582+0000","last_unstale":"2024-04-26T19:50:38.653594+0000","last_undegraded":"2024-04-26T19:50:38.653594+0000","last_fullsized":"2024-04-26T19:50:38.653594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653995+0000","last_change":"2024-04-26T19:50:31.120475+0000","last_active":"2024-04-26T19:50:38.653995+0000","last_peered":"2024-04-26T19:50:38.653995+0000","last_clean":"2024-04-26T19:50:38.653995+0000","last_became_active":"2024-04-26T19:50:31.118576+0000","last_became_peered":"2024-04-26T19:50:31.118576+0000","last_unstale":"2024-04-26T19:50:38.653995+0000","last_undegraded":"2024-04-26T19:50:38.653995+0000","last_fullsized":"2024-04-26T19:50:38.653995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.291653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130241+0000","last_change":"2024-04-26T19:50:31.111907+0000","last_active":"2024-04-26T19:50:37.130241+0000","last_peered":"2024-04-26T19:50:37.130241+0000","last_clean":"2024-04-26T19:50:37.130241+0000","last_became_active":"2024-04-26T19:50:31.111623+0000","last_became_peered":"2024-04-26T19:50:31.111623+0000","last_unstale":"2024-04-26T19:50:37.130241+0000","last_undegraded":"2024-04-26T19:50:37.130241+0000","last_fullsized":"2024-04-26T19:50:37.130241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652974+0000","last_change":"2024-04-26T19:50:31.112347+0000","last_active":"2024-04-26T19:50:38.652974+0000","last_peered":"2024-04-26T19:50:38.652974+0000","last_clean":"2024-04-26T19:50:38.652974+0000","last_became_active":"2024-04-26T19:50:31.112074+0000","last_became_peered":"2024-04-26T19:50:31.112074+0000","last_unstale":"2024-04-26T19:50:38.652974+0000","last_undegraded":"2024-04-26T19:50:38.652974+0000","last_fullsized":"2024-04-26T19:50:38.652974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654050+0000","last_change":"2024-04-26T19:50:31.115255+0000","last_active":"2024-04-26T19:50:38.654050+0000","last_peered":"2024-04-26T19:50:38.654050+0000","last_clean":"2024-04-26T19:50:38.654050+0000","last_became_active":"2024-04-26T19:50:31.114991+0000","last_became_peered":"2024-04-26T19:50:31.114991+0000","last_unstale":"2024-04-26T19:50:38.654050+0000","last_undegraded":"2024-04-26T19:50:38.654050+0000","last_fullsized":"2024-04-26T19:50:38.654050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:39.941264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653558+0000","last_change":"2024-04-26T19:50:31.111763+0000","last_active":"2024-04-26T19:50:38.653558+0000","last_peered":"2024-04-26T19:50:38.653558+0000","last_clean":"2024-04-26T19:50:38.653558+0000","last_became_active":"2024-04-26T19:50:31.111487+0000","last_became_peered":"2024-04-26T19:50:31.111487+0000","last_unstale":"2024-04-26T19:50:38.653558+0000","last_undegraded":"2024-04-26T19:50:38.653558+0000","last_fullsized":"2024-04-26T19:50:38.653558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:13.577254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652454+0000","last_change":"2024-04-26T19:50:31.120637+0000","last_active":"2024-04-26T19:50:38.652454+0000","last_peered":"2024-04-26T19:50:38.652454+0000","last_clean":"2024-04-26T19:50:38.652454+0000","last_became_active":"2024-04-26T19:50:31.120388+0000","last_became_peered":"2024-04-26T19:50:31.120388+0000","last_unstale":"2024-04-26T19:50:38.652454+0000","last_undegraded":"2024-04-26T19:50:38.652454+0000","last_fullsized":"2024-04-26T19:50:38.652454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299955+0000","last_change":"2024-04-26T19:50:31.104484+0000","last_active":"2024-04-26T19:50:38.299955+0000","last_peered":"2024-04-26T19:50:38.299955+0000","last_clean":"2024-04-26T19:50:38.299955+0000","last_became_active":"2024-04-26T19:50:31.104172+0000","last_became_peered":"2024-04-26T19:50:31.104172+0000","last_unstale":"2024-04-26T19:50:38.299955+0000","last_undegraded":"2024-04-26T19:50:38.299955+0000","last_fullsized":"2024-04-26T19:50:38.299955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997884+0000","last_change":"2024-04-26T19:50:31.110923+0000","last_active":"2024-04-26T19:50:37.997884+0000","last_peered":"2024-04-26T19:50:37.997884+0000","last_clean":"2024-04-26T19:50:37.997884+0000","last_became_active":"2024-04-26T19:50:31.110601+0000","last_became_peered":"2024-04-26T19:50:31.110601+0000","last_unstale":"2024-04-26T19:50:37.997884+0000","last_undegraded":"2024-04-26T19:50:37.997884+0000","last_fullsized":"2024-04-26T19:50:37.997884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652978+0000","last_change":"2024-04-26T19:50:31.112465+0000","last_active":"2024-04-26T19:50:38.652978+0000","last_peered":"2024-04-26T19:50:38.652978+0000","last_clean":"2024-04-26T19:50:38.652978+0000","last_became_active":"2024-04-26T19:50:31.112186+0000","last_became_peered":"2024-04-26T19:50:31.112186+0000","last_unstale":"2024-04-26T19:50:38.652978+0000","last_undegraded":"2024-04-26T19:50:38.652978+0000","last_fullsized":"2024-04-26T19:50:38.652978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:10.399495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:50:54.141 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:50:54.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.274+0000 7f4d2e7f0640 1 Processor -- start 2024-04-26T19:50:54.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.274+0000 7f4d2e7f0640 1 -- start start 2024-04-26T19:50:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d2e7f0640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d280b4220 0x7f4d281630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d2e7f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d28160770 0x7f4d28163610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d2e7f0640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d28144ca0 con 0x7f4d28160390 2024-04-26T19:50:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d2e7f0640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f4d28163b50 con 0x7f4d280b4220 2024-04-26T19:50:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d2e7f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4d28163e00 con 0x7f4d28160770 2024-04-26T19:50:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d277fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4d280b4220 0x7f4d281630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d27fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4d28160390 0x7f4d280b3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46242/0 (socket says 172.21.15.79:46242) 2024-04-26T19:50:54.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d27fff640 1 -- 172.21.15.79:0/864510591 learned_addr learned my addr 172.21.15.79:0/864510591 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:54.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d2cd8d640 1 --2- 172.21.15.79:0/864510591 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d28160770 0x7f4d28163610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d277fe640 1 -- 172.21.15.79:0/864510591 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d28160770 msgr2=0x7f4d28163610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:54.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d277fe640 1 --2- 172.21.15.79:0/864510591 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d28160770 0x7f4d28163610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d277fe640 1 -- 172.21.15.79:0/864510591 >> v1:172.21.15.79:6789/0 conn(0x7f4d28160390 legacy=0x7f4d280b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d26ffd640 1 -- 172.21.15.79:0/864510591 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 797963870 0 0) 0x7f4d28144ca0 con 0x7f4d28160390 2024-04-26T19:50:54.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d277fe640 1 -- 172.21.15.79:0/864510591 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d28144f30 con 0x7f4d280b4220 2024-04-26T19:50:54.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.275+0000 7f4d2cd8d640 1 --2- 172.21.15.79:0/864510591 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d28160770 0x7f4d28163610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:54.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.276+0000 7f4d277fe640 1 --2- 172.21.15.79:0/864510591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d280b4220 0x7f4d281630d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0097a0 tx=0x7f4d1c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2153de580d05f2ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:54.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.276+0000 7f4d26ffd640 1 -- 172.21.15.79:0/864510591 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c037070 con 0x7f4d280b4220 2024-04-26T19:50:54.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.276+0000 7f4d26ffd640 1 -- 172.21.15.79:0/864510591 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1c02ae50 con 0x7f4d280b4220 2024-04-26T19:50:54.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.276+0000 7f4d26ffd640 1 -- 172.21.15.79:0/864510591 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c032660 con 0x7f4d280b4220 2024-04-26T19:50:54.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.277+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/864510591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d280b4220 msgr2=0x7f4d281630d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.277+0000 7f4d2e7f0640 1 --2- 172.21.15.79:0/864510591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d280b4220 0x7f4d281630d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0097a0 tx=0x7f4d1c02a100 comp rx=0 tx=0).stop 2024-04-26T19:50:54.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.277+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/864510591 shutdown_connections 2024-04-26T19:50:54.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.277+0000 7f4d2e7f0640 1 --2- 172.21.15.79:0/864510591 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d28160770 0x7f4d28163610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.277+0000 7f4d2e7f0640 1 --2- 172.21.15.79:0/864510591 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d280b4220 0x7f4d281630d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.277+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/864510591 >> 172.21.15.79:0/864510591 conn(0x7f4d280b2720 msgr2=0x7f4d28070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:54.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.277+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/864510591 shutdown_connections 2024-04-26T19:50:54.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.277+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/864510591 wait complete. 2024-04-26T19:50:54.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.278+0000 7f4d2e7f0640 1 Processor -- start 2024-04-26T19:50:54.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.278+0000 7f4d2e7f0640 1 -- start start 2024-04-26T19:50:54.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d2e7f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d280b4220 0x7f4d2815c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d2e7f0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d28160390 0x7f4d2806f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d2e7f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28160770 0x7f4d2806faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d2e7f0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4d28145a10 con 0x7f4d28160390 2024-04-26T19:50:54.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d2e7f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4d281454f0 con 0x7f4d28160770 2024-04-26T19:50:54.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d2e7f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4d28145b90 con 0x7f4d280b4220 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d277fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d28160390 0x7f4d2806f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d277fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d28160390 0x7f4d2806f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38694/0 (socket says 172.21.15.79:38694) 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d277fe640 1 -- 172.21.15.79:0/2327933048 learned_addr learned my addr 172.21.15.79:0/2327933048 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d277fe640 1 -- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d280b4220 msgr2=0x7f4d2815c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d27fff640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d280b4220 0x7f4d2815c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d2cd8d640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28160770 0x7f4d2806faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d277fe640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d280b4220 0x7f4d2815c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d277fe640 1 -- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28160770 msgr2=0x7f4d2806faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d277fe640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28160770 0x7f4d2806faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d277fe640 1 -- 172.21.15.79:0/2327933048 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2815cf20 con 0x7f4d28160390 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d27fff640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d280b4220 0x7f4d2815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d2cd8d640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28160770 0x7f4d2806faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d277fe640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d28160390 0x7f4d2806f5b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0097a0 tx=0x7f4d1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.279+0000 7f4d24ff9640 1 -- 172.21.15.79:0/2327933048 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c037040 con 0x7f4d28160390 2024-04-26T19:50:54.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.280+0000 7f4d24ff9640 1 -- 172.21.15.79:0/2327933048 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d1c004250 con 0x7f4d28160390 2024-04-26T19:50:54.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.280+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d280af400 con 0x7f4d28160390 2024-04-26T19:50:54.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.280+0000 7f4d24ff9640 1 -- 172.21.15.79:0/2327933048 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1c003610 con 0x7f4d28160390 2024-04-26T19:50:54.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.280+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2815a4b0 con 0x7f4d28160390 2024-04-26T19:50:54.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.281+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d280506c0 con 0x7f4d28160390 2024-04-26T19:50:54.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.281+0000 7f4d24ff9640 1 -- 172.21.15.79:0/2327933048 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d1c02ad10 con 0x7f4d28160390 2024-04-26T19:50:54.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.281+0000 7f4d24ff9640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4d1406c8f0 0x7f4d1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.281+0000 7f4d27fff640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4d1406c8f0 0x7f4d1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.281+0000 7f4d24ff9640 1 -- 172.21.15.79:0/2327933048 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4d1c0acae0 con 0x7f4d28160390 2024-04-26T19:50:54.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.282+0000 7f4d27fff640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4d1406c8f0 0x7f4d1406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000f60 tx=0x7f4d18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:54.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.285+0000 7f4d24ff9640 1 -- 172.21.15.79:0/2327933048 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4d1c07ad80 con 0x7f4d28160390 2024-04-26T19:50:54.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.457+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4d280af6e0 con 0x7f4d1406c8f0 2024-04-26T19:50:54.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.465+0000 7f4d24ff9640 1 -- 172.21.15.79:0/2327933048 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f4d280af6e0 con 0x7f4d1406c8f0 2024-04-26T19:50:54.469 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:54.471 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:50:54.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4d1406c8f0 msgr2=0x7f4d1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4d1406c8f0 0x7f4d1406edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4d18000f60 tx=0x7f4d18009040 comp rx=0 tx=0).stop 2024-04-26T19:50:54.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d28160390 msgr2=0x7f4d2806f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d28160390 0x7f4d2806f5b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c0097a0 tx=0x7f4d1c0046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 shutdown_connections 2024-04-26T19:50:54.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4d1406c8f0 0x7f4d1406edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4d28160770 0x7f4d2806faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4d28160390 0x7f4d2806f5b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 --2- 172.21.15.79:0/2327933048 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4d280b4220 0x7f4d2815c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 >> 172.21.15.79:0/2327933048 conn(0x7f4d280b2720 msgr2=0x7f4d2806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:54.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 shutdown_connections 2024-04-26T19:50:54.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.468+0000 7f4d2e7f0640 1 -- 172.21.15.79:0/2327933048 wait complete. 2024-04-26T19:50:54.480 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-26T19:50:54.256923+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219460,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755260,"statfs":{"total":773094113280,"available":772869386240,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002527"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300284+0000","last_change":"2024-04-26T19:50:30.105908+0000","last_active":"2024-04-26T19:50:38.300284+0000","last_peered":"2024-04-26T19:50:38.300284+0000","last_clean":"2024-04-26T19:50:38.300284+0000","last_became_active":"2024-04-26T19:50:30.105650+0000","last_became_peered":"2024-04-26T19:50:30.105650+0000","last_unstale":"2024-04-26T19:50:38.300284+0000","last_undegraded":"2024-04-26T19:50:38.300284+0000","last_fullsized":"2024-04-26T19:50:38.300284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:25.896804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131172+0000","last_change":"2024-04-26T19:50:31.110936+0000","last_active":"2024-04-26T19:50:37.131172+0000","last_peered":"2024-04-26T19:50:37.131172+0000","last_clean":"2024-04-26T19:50:37.131172+0000","last_became_active":"2024-04-26T19:50:31.110602+0000","last_became_peered":"2024-04-26T19:50:31.110602+0000","last_unstale":"2024-04-26T19:50:37.131172+0000","last_undegraded":"2024-04-26T19:50:37.131172+0000","last_fullsized":"2024-04-26T19:50:37.131172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130254+0000","last_change":"2024-04-26T19:50:35.626950+0000","last_active":"2024-04-26T19:50:37.130254+0000","last_peered":"2024-04-26T19:50:37.130254+0000","last_clean":"2024-04-26T19:50:37.130254+0000","last_became_active":"2024-04-26T19:50:35.626309+0000","last_became_peered":"2024-04-26T19:50:35.626309+0000","last_unstale":"2024-04-26T19:50:37.130254+0000","last_undegraded":"2024-04-26T19:50:37.130254+0000","last_fullsized":"2024-04-26T19:50:37.130254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:30:42.055139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131089+0000","last_change":"2024-04-26T19:50:30.112742+0000","last_active":"2024-04-26T19:50:37.131089+0000","last_peered":"2024-04-26T19:50:37.131089+0000","last_clean":"2024-04-26T19:50:37.131089+0000","last_became_active":"2024-04-26T19:50:30.112460+0000","last_became_peered":"2024-04-26T19:50:30.112460+0000","last_unstale":"2024-04-26T19:50:37.131089+0000","last_undegraded":"2024-04-26T19:50:37.131089+0000","last_fullsized":"2024-04-26T19:50:37.131089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:21.153340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300125+0000","last_change":"2024-04-26T19:50:31.114135+0000","last_active":"2024-04-26T19:50:38.300125+0000","last_peered":"2024-04-26T19:50:38.300125+0000","last_clean":"2024-04-26T19:50:38.300125+0000","last_became_active":"2024-04-26T19:50:31.113834+0000","last_became_peered":"2024-04-26T19:50:31.113834+0000","last_unstale":"2024-04-26T19:50:38.300125+0000","last_undegraded":"2024-04-26T19:50:38.300125+0000","last_fullsized":"2024-04-26T19:50:38.300125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129662+0000","last_change":"2024-04-26T19:50:35.141805+0000","last_active":"2024-04-26T19:50:37.129662+0000","last_peered":"2024-04-26T19:50:37.129662+0000","last_clean":"2024-04-26T19:50:37.129662+0000","last_became_active":"2024-04-26T19:50:35.141362+0000","last_became_peered":"2024-04-26T19:50:35.141362+0000","last_unstale":"2024-04-26T19:50:37.129662+0000","last_undegraded":"2024-04-26T19:50:37.129662+0000","last_fullsized":"2024-04-26T19:50:37.129662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:10.325286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653474+0000","last_change":"2024-04-26T19:50:30.107616+0000","last_active":"2024-04-26T19:50:38.653474+0000","last_peered":"2024-04-26T19:50:38.653474+0000","last_clean":"2024-04-26T19:50:38.653474+0000","last_became_active":"2024-04-26T19:50:30.107304+0000","last_became_peered":"2024-04-26T19:50:30.107304+0000","last_unstale":"2024-04-26T19:50:38.653474+0000","last_undegraded":"2024-04-26T19:50:38.653474+0000","last_fullsized":"2024-04-26T19:50:38.653474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:12.144175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652574+0000","last_change":"2024-04-26T19:50:31.114725+0000","last_active":"2024-04-26T19:50:38.652574+0000","last_peered":"2024-04-26T19:50:38.652574+0000","last_clean":"2024-04-26T19:50:38.652574+0000","last_became_active":"2024-04-26T19:50:31.114444+0000","last_became_peered":"2024-04-26T19:50:31.114444+0000","last_unstale":"2024-04-26T19:50:38.652574+0000","last_undegraded":"2024-04-26T19:50:38.652574+0000","last_fullsized":"2024-04-26T19:50:38.652574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129888+0000","last_change":"2024-04-26T19:50:35.145133+0000","last_active":"2024-04-26T19:50:37.129888+0000","last_peered":"2024-04-26T19:50:37.129888+0000","last_clean":"2024-04-26T19:50:37.129888+0000","last_became_active":"2024-04-26T19:50:35.144310+0000","last_became_peered":"2024-04-26T19:50:35.144310+0000","last_unstale":"2024-04-26T19:50:37.129888+0000","last_undegraded":"2024-04-26T19:50:37.129888+0000","last_fullsized":"2024-04-26T19:50:37.129888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:35.220332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652453+0000","last_change":"2024-04-26T19:50:30.115029+0000","last_active":"2024-04-26T19:50:38.652453+0000","last_peered":"2024-04-26T19:50:38.652453+0000","last_clean":"2024-04-26T19:50:38.652453+0000","last_became_active":"2024-04-26T19:50:30.114719+0000","last_became_peered":"2024-04-26T19:50:30.114719+0000","last_unstale":"2024-04-26T19:50:38.652453+0000","last_undegraded":"2024-04-26T19:50:38.652453+0000","last_fullsized":"2024-04-26T19:50:38.652453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:28.557299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654620+0000","last_change":"2024-04-26T19:50:31.114802+0000","last_active":"2024-04-26T19:50:38.654620+0000","last_peered":"2024-04-26T19:50:38.654620+0000","last_clean":"2024-04-26T19:50:38.654620+0000","last_became_active":"2024-04-26T19:50:31.114489+0000","last_became_peered":"2024-04-26T19:50:31.114489+0000","last_unstale":"2024-04-26T19:50:38.654620+0000","last_undegraded":"2024-04-26T19:50:38.654620+0000","last_fullsized":"2024-04-26T19:50:38.654620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129602+0000","last_change":"2024-04-26T19:50:35.147238+0000","last_active":"2024-04-26T19:50:37.129602+0000","last_peered":"2024-04-26T19:50:37.129602+0000","last_clean":"2024-04-26T19:50:37.129602+0000","last_became_active":"2024-04-26T19:50:35.146793+0000","last_became_peered":"2024-04-26T19:50:35.146793+0000","last_unstale":"2024-04-26T19:50:37.129602+0000","last_undegraded":"2024-04-26T19:50:37.129602+0000","last_fullsized":"2024-04-26T19:50:37.129602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:14.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654487+0000","last_change":"2024-04-26T19:50:31.104842+0000","last_active":"2024-04-26T19:50:38.654487+0000","last_peered":"2024-04-26T19:50:38.654487+0000","last_clean":"2024-04-26T19:50:38.654487+0000","last_became_active":"2024-04-26T19:50:31.104587+0000","last_became_peered":"2024-04-26T19:50:31.104587+0000","last_unstale":"2024-04-26T19:50:38.654487+0000","last_undegraded":"2024-04-26T19:50:38.654487+0000","last_fullsized":"2024-04-26T19:50:38.654487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653208+0000","last_change":"2024-04-26T19:50:30.112050+0000","last_active":"2024-04-26T19:50:38.653208+0000","last_peered":"2024-04-26T19:50:38.653208+0000","last_clean":"2024-04-26T19:50:38.653208+0000","last_became_active":"2024-04-26T19:50:30.111754+0000","last_became_peered":"2024-04-26T19:50:30.111754+0000","last_unstale":"2024-04-26T19:50:38.653208+0000","last_undegraded":"2024-04-26T19:50:38.653208+0000","last_fullsized":"2024-04-26T19:50:38.653208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130351+0000","last_change":"2024-04-26T19:50:35.145983+0000","last_active":"2024-04-26T19:50:37.130351+0000","last_peered":"2024-04-26T19:50:37.130351+0000","last_clean":"2024-04-26T19:50:37.130351+0000","last_became_active":"2024-04-26T19:50:35.144815+0000","last_became_peered":"2024-04-26T19:50:35.144815+0000","last_unstale":"2024-04-26T19:50:37.130351+0000","last_undegraded":"2024-04-26T19:50:37.130351+0000","last_fullsized":"2024-04-26T19:50:37.130351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130515+0000","last_change":"2024-04-26T19:50:31.109267+0000","last_active":"2024-04-26T19:50:37.130515+0000","last_peered":"2024-04-26T19:50:37.130515+0000","last_clean":"2024-04-26T19:50:37.130515+0000","last_became_active":"2024-04-26T19:50:31.108771+0000","last_became_peered":"2024-04-26T19:50:31.108771+0000","last_unstale":"2024-04-26T19:50:37.130515+0000","last_undegraded":"2024-04-26T19:50:37.130515+0000","last_fullsized":"2024-04-26T19:50:37.130515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997666+0000","last_change":"2024-04-26T19:50:30.110310+0000","last_active":"2024-04-26T19:50:37.997666+0000","last_peered":"2024-04-26T19:50:37.997666+0000","last_clean":"2024-04-26T19:50:37.997666+0000","last_became_active":"2024-04-26T19:50:30.109819+0000","last_became_peered":"2024-04-26T19:50:30.109819+0000","last_unstale":"2024-04-26T19:50:37.997666+0000","last_undegraded":"2024-04-26T19:50:37.997666+0000","last_fullsized":"2024-04-26T19:50:37.997666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:41.838039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130058+0000","last_change":"2024-04-26T19:50:35.143564+0000","last_active":"2024-04-26T19:50:37.130058+0000","last_peered":"2024-04-26T19:50:37.130058+0000","last_clean":"2024-04-26T19:50:37.130058+0000","last_became_active":"2024-04-26T19:50:35.143119+0000","last_became_peered":"2024-04-26T19:50:35.143119+0000","last_unstale":"2024-04-26T19:50:37.130058+0000","last_undegraded":"2024-04-26T19:50:37.130058+0000","last_fullsized":"2024-04-26T19:50:37.130058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:11.052818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130395+0000","last_change":"2024-04-26T19:50:31.112367+0000","last_active":"2024-04-26T19:50:37.130395+0000","last_peered":"2024-04-26T19:50:37.130395+0000","last_clean":"2024-04-26T19:50:37.130395+0000","last_became_active":"2024-04-26T19:50:31.112070+0000","last_became_peered":"2024-04-26T19:50:31.112070+0000","last_unstale":"2024-04-26T19:50:37.130395+0000","last_undegraded":"2024-04-26T19:50:37.130395+0000","last_fullsized":"2024-04-26T19:50:37.130395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997401+0000","last_change":"2024-04-26T19:50:30.105988+0000","last_active":"2024-04-26T19:50:37.997401+0000","last_peered":"2024-04-26T19:50:37.997401+0000","last_clean":"2024-04-26T19:50:37.997401+0000","last_became_active":"2024-04-26T19:50:30.103503+0000","last_became_peered":"2024-04-26T19:50:30.103503+0000","last_unstale":"2024-04-26T19:50:37.997401+0000","last_undegraded":"2024-04-26T19:50:37.997401+0000","last_fullsized":"2024-04-26T19:50:37.997401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997963+0000","last_change":"2024-04-26T19:50:35.626255+0000","last_active":"2024-04-26T19:50:37.997963+0000","last_peered":"2024-04-26T19:50:37.997963+0000","last_clean":"2024-04-26T19:50:37.997963+0000","last_became_active":"2024-04-26T19:50:35.625845+0000","last_became_peered":"2024-04-26T19:50:35.625845+0000","last_unstale":"2024-04-26T19:50:37.997963+0000","last_undegraded":"2024-04-26T19:50:37.997963+0000","last_fullsized":"2024-04-26T19:50:37.997963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:02.874670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129835+0000","last_change":"2024-04-26T19:50:31.106338+0000","last_active":"2024-04-26T19:50:37.129835+0000","last_peered":"2024-04-26T19:50:37.129835+0000","last_clean":"2024-04-26T19:50:37.129835+0000","last_became_active":"2024-04-26T19:50:31.106097+0000","last_became_peered":"2024-04-26T19:50:31.106097+0000","last_unstale":"2024-04-26T19:50:37.129835+0000","last_undegraded":"2024-04-26T19:50:37.129835+0000","last_fullsized":"2024-04-26T19:50:37.129835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:13.140788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654064+0000","last_change":"2024-04-26T19:50:30.114014+0000","last_active":"2024-04-26T19:50:38.654064+0000","last_peered":"2024-04-26T19:50:38.654064+0000","last_clean":"2024-04-26T19:50:38.654064+0000","last_became_active":"2024-04-26T19:50:30.113720+0000","last_became_peered":"2024-04-26T19:50:30.113720+0000","last_unstale":"2024-04-26T19:50:38.654064+0000","last_undegraded":"2024-04-26T19:50:38.654064+0000","last_fullsized":"2024-04-26T19:50:38.654064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130140+0000","last_change":"2024-04-26T19:50:35.144698+0000","last_active":"2024-04-26T19:50:37.130140+0000","last_peered":"2024-04-26T19:50:37.130140+0000","last_clean":"2024-04-26T19:50:37.130140+0000","last_became_active":"2024-04-26T19:50:35.144169+0000","last_became_peered":"2024-04-26T19:50:35.144169+0000","last_unstale":"2024-04-26T19:50:37.130140+0000","last_undegraded":"2024-04-26T19:50:37.130140+0000","last_fullsized":"2024-04-26T19:50:37.130140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300094+0000","last_change":"2024-04-26T19:50:31.105626+0000","last_active":"2024-04-26T19:50:38.300094+0000","last_peered":"2024-04-26T19:50:38.300094+0000","last_clean":"2024-04-26T19:50:38.300094+0000","last_became_active":"2024-04-26T19:50:31.105369+0000","last_became_peered":"2024-04-26T19:50:31.105369+0000","last_unstale":"2024-04-26T19:50:38.300094+0000","last_undegraded":"2024-04-26T19:50:38.300094+0000","last_fullsized":"2024-04-26T19:50:38.300094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654017+0000","last_change":"2024-04-26T19:50:30.104092+0000","last_active":"2024-04-26T19:50:38.654017+0000","last_peered":"2024-04-26T19:50:38.654017+0000","last_clean":"2024-04-26T19:50:38.654017+0000","last_became_active":"2024-04-26T19:50:30.103848+0000","last_became_peered":"2024-04-26T19:50:30.103848+0000","last_unstale":"2024-04-26T19:50:38.654017+0000","last_undegraded":"2024-04-26T19:50:38.654017+0000","last_fullsized":"2024-04-26T19:50:38.654017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129689+0000","last_change":"2024-04-26T19:50:35.624499+0000","last_active":"2024-04-26T19:50:37.129689+0000","last_peered":"2024-04-26T19:50:37.129689+0000","last_clean":"2024-04-26T19:50:37.129689+0000","last_became_active":"2024-04-26T19:50:35.624026+0000","last_became_peered":"2024-04-26T19:50:35.624026+0000","last_unstale":"2024-04-26T19:50:37.129689+0000","last_undegraded":"2024-04-26T19:50:37.129689+0000","last_fullsized":"2024-04-26T19:50:37.129689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:46.591736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131049+0000","last_change":"2024-04-26T19:50:31.116347+0000","last_active":"2024-04-26T19:50:37.131049+0000","last_peered":"2024-04-26T19:50:37.131049+0000","last_clean":"2024-04-26T19:50:37.131049+0000","last_became_active":"2024-04-26T19:50:31.116037+0000","last_became_peered":"2024-04-26T19:50:31.116037+0000","last_unstale":"2024-04-26T19:50:37.131049+0000","last_undegraded":"2024-04-26T19:50:37.131049+0000","last_fullsized":"2024-04-26T19:50:37.131049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997937+0000","last_change":"2024-04-26T19:50:30.105663+0000","last_active":"2024-04-26T19:50:37.997937+0000","last_peered":"2024-04-26T19:50:37.997937+0000","last_clean":"2024-04-26T19:50:37.997937+0000","last_became_active":"2024-04-26T19:50:30.105387+0000","last_became_peered":"2024-04-26T19:50:30.105387+0000","last_unstale":"2024-04-26T19:50:37.997937+0000","last_undegraded":"2024-04-26T19:50:37.997937+0000","last_fullsized":"2024-04-26T19:50:37.997937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130198+0000","last_change":"2024-04-26T19:50:35.625444+0000","last_active":"2024-04-26T19:50:37.130198+0000","last_peered":"2024-04-26T19:50:37.130198+0000","last_clean":"2024-04-26T19:50:37.130198+0000","last_became_active":"2024-04-26T19:50:35.625054+0000","last_became_peered":"2024-04-26T19:50:35.625054+0000","last_unstale":"2024-04-26T19:50:37.130198+0000","last_undegraded":"2024-04-26T19:50:37.130198+0000","last_fullsized":"2024-04-26T19:50:37.130198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:07.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997757+0000","last_change":"2024-04-26T19:50:31.109145+0000","last_active":"2024-04-26T19:50:37.997757+0000","last_peered":"2024-04-26T19:50:37.997757+0000","last_clean":"2024-04-26T19:50:37.997757+0000","last_became_active":"2024-04-26T19:50:31.108812+0000","last_became_peered":"2024-04-26T19:50:31.108812+0000","last_unstale":"2024-04-26T19:50:37.997757+0000","last_undegraded":"2024-04-26T19:50:37.997757+0000","last_fullsized":"2024-04-26T19:50:37.997757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653987+0000","last_change":"2024-04-26T19:50:30.112025+0000","last_active":"2024-04-26T19:50:38.653987+0000","last_peered":"2024-04-26T19:50:38.653987+0000","last_clean":"2024-04-26T19:50:38.653987+0000","last_became_active":"2024-04-26T19:50:30.111777+0000","last_became_peered":"2024-04-26T19:50:30.111777+0000","last_unstale":"2024-04-26T19:50:38.653987+0000","last_undegraded":"2024-04-26T19:50:38.653987+0000","last_fullsized":"2024-04-26T19:50:38.653987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:41.838039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299918+0000","last_change":"2024-04-26T19:50:35.623970+0000","last_active":"2024-04-26T19:50:38.299918+0000","last_peered":"2024-04-26T19:50:38.299918+0000","last_clean":"2024-04-26T19:50:38.299918+0000","last_became_active":"2024-04-26T19:50:35.623472+0000","last_became_peered":"2024-04-26T19:50:35.623472+0000","last_unstale":"2024-04-26T19:50:38.299918+0000","last_undegraded":"2024-04-26T19:50:38.299918+0000","last_fullsized":"2024-04-26T19:50:38.299918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654678+0000","last_change":"2024-04-26T19:50:31.120338+0000","last_active":"2024-04-26T19:50:38.654678+0000","last_peered":"2024-04-26T19:50:38.654678+0000","last_clean":"2024-04-26T19:50:38.654678+0000","last_became_active":"2024-04-26T19:50:31.120074+0000","last_became_peered":"2024-04-26T19:50:31.120074+0000","last_unstale":"2024-04-26T19:50:38.654678+0000","last_undegraded":"2024-04-26T19:50:38.654678+0000","last_fullsized":"2024-04-26T19:50:38.654678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997828+0000","last_change":"2024-04-26T19:50:30.106832+0000","last_active":"2024-04-26T19:50:37.997828+0000","last_peered":"2024-04-26T19:50:37.997828+0000","last_clean":"2024-04-26T19:50:37.997828+0000","last_became_active":"2024-04-26T19:50:30.106553+0000","last_became_peered":"2024-04-26T19:50:30.106553+0000","last_unstale":"2024-04-26T19:50:37.997828+0000","last_undegraded":"2024-04-26T19:50:37.997828+0000","last_fullsized":"2024-04-26T19:50:37.997828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997421+0000","last_change":"2024-04-26T19:50:35.149679+0000","last_active":"2024-04-26T19:50:37.997421+0000","last_peered":"2024-04-26T19:50:37.997421+0000","last_clean":"2024-04-26T19:50:37.997421+0000","last_became_active":"2024-04-26T19:50:35.149304+0000","last_became_peered":"2024-04-26T19:50:35.149304+0000","last_unstale":"2024-04-26T19:50:37.997421+0000","last_undegraded":"2024-04-26T19:50:37.997421+0000","last_fullsized":"2024-04-26T19:50:37.997421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130385+0000","last_change":"2024-04-26T19:50:31.105502+0000","last_active":"2024-04-26T19:50:37.130385+0000","last_peered":"2024-04-26T19:50:37.130385+0000","last_clean":"2024-04-26T19:50:37.130385+0000","last_became_active":"2024-04-26T19:50:31.105047+0000","last_became_peered":"2024-04-26T19:50:31.105047+0000","last_unstale":"2024-04-26T19:50:37.130385+0000","last_undegraded":"2024-04-26T19:50:37.130385+0000","last_fullsized":"2024-04-26T19:50:37.130385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:20.874838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653576+0000","last_change":"2024-04-26T19:50:30.110096+0000","last_active":"2024-04-26T19:50:38.653576+0000","last_peered":"2024-04-26T19:50:38.653576+0000","last_clean":"2024-04-26T19:50:38.653576+0000","last_became_active":"2024-04-26T19:50:30.109131+0000","last_became_peered":"2024-04-26T19:50:30.109131+0000","last_unstale":"2024-04-26T19:50:38.653576+0000","last_undegraded":"2024-04-26T19:50:38.653576+0000","last_fullsized":"2024-04-26T19:50:38.653576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997740+0000","last_change":"2024-04-26T19:50:35.148072+0000","last_active":"2024-04-26T19:50:37.997740+0000","last_peered":"2024-04-26T19:50:37.997740+0000","last_clean":"2024-04-26T19:50:37.997740+0000","last_became_active":"2024-04-26T19:50:35.147697+0000","last_became_peered":"2024-04-26T19:50:35.147697+0000","last_unstale":"2024-04-26T19:50:37.997740+0000","last_undegraded":"2024-04-26T19:50:37.997740+0000","last_fullsized":"2024-04-26T19:50:37.997740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997366+0000","last_change":"2024-04-26T19:50:31.108339+0000","last_active":"2024-04-26T19:50:37.997366+0000","last_peered":"2024-04-26T19:50:37.997366+0000","last_clean":"2024-04-26T19:50:37.997366+0000","last_became_active":"2024-04-26T19:50:31.108056+0000","last_became_peered":"2024-04-26T19:50:31.108056+0000","last_unstale":"2024-04-26T19:50:37.997366+0000","last_undegraded":"2024-04-26T19:50:37.997366+0000","last_fullsized":"2024-04-26T19:50:37.997366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536841+0000","last_change":"2024-04-26T19:50:30.111350+0000","last_active":"2024-04-26T19:50:38.536841+0000","last_peered":"2024-04-26T19:50:38.536841+0000","last_clean":"2024-04-26T19:50:38.536841+0000","last_became_active":"2024-04-26T19:50:30.111070+0000","last_became_peered":"2024-04-26T19:50:30.111070+0000","last_unstale":"2024-04-26T19:50:38.536841+0000","last_undegraded":"2024-04-26T19:50:38.536841+0000","last_fullsized":"2024-04-26T19:50:38.536841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.998002+0000","last_change":"2024-04-26T19:50:35.146694+0000","last_active":"2024-04-26T19:50:37.998002+0000","last_peered":"2024-04-26T19:50:37.998002+0000","last_clean":"2024-04-26T19:50:37.998002+0000","last_became_active":"2024-04-26T19:50:35.145805+0000","last_became_peered":"2024-04-26T19:50:35.145805+0000","last_unstale":"2024-04-26T19:50:37.998002+0000","last_undegraded":"2024-04-26T19:50:37.998002+0000","last_fullsized":"2024-04-26T19:50:37.998002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:01.350601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299548+0000","last_change":"2024-04-26T19:50:31.103279+0000","last_active":"2024-04-26T19:50:38.299548+0000","last_peered":"2024-04-26T19:50:38.299548+0000","last_clean":"2024-04-26T19:50:38.299548+0000","last_became_active":"2024-04-26T19:50:31.102858+0000","last_became_peered":"2024-04-26T19:50:31.102858+0000","last_unstale":"2024-04-26T19:50:38.299548+0000","last_undegraded":"2024-04-26T19:50:38.299548+0000","last_fullsized":"2024-04-26T19:50:38.299548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:56.497144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653771+0000","last_change":"2024-04-26T19:50:30.109372+0000","last_active":"2024-04-26T19:50:38.653771+0000","last_peered":"2024-04-26T19:50:38.653771+0000","last_clean":"2024-04-26T19:50:38.653771+0000","last_became_active":"2024-04-26T19:50:30.109087+0000","last_became_peered":"2024-04-26T19:50:30.109087+0000","last_unstale":"2024-04-26T19:50:38.653771+0000","last_undegraded":"2024-04-26T19:50:38.653771+0000","last_fullsized":"2024-04-26T19:50:38.653771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:13.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300021+0000","last_change":"2024-04-26T19:50:35.623867+0000","last_active":"2024-04-26T19:50:38.300021+0000","last_peered":"2024-04-26T19:50:38.300021+0000","last_clean":"2024-04-26T19:50:38.300021+0000","last_became_active":"2024-04-26T19:50:35.623416+0000","last_became_peered":"2024-04-26T19:50:35.623416+0000","last_unstale":"2024-04-26T19:50:38.300021+0000","last_undegraded":"2024-04-26T19:50:38.300021+0000","last_fullsized":"2024-04-26T19:50:38.300021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:02.874670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130394+0000","last_change":"2024-04-26T19:50:31.113014+0000","last_active":"2024-04-26T19:50:37.130394+0000","last_peered":"2024-04-26T19:50:37.130394+0000","last_clean":"2024-04-26T19:50:37.130394+0000","last_became_active":"2024-04-26T19:50:31.112753+0000","last_became_peered":"2024-04-26T19:50:31.112753+0000","last_unstale":"2024-04-26T19:50:37.130394+0000","last_undegraded":"2024-04-26T19:50:37.130394+0000","last_fullsized":"2024-04-26T19:50:37.130394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653084+0000","last_change":"2024-04-26T19:50:30.110312+0000","last_active":"2024-04-26T19:50:38.653084+0000","last_peered":"2024-04-26T19:50:38.653084+0000","last_clean":"2024-04-26T19:50:38.653084+0000","last_became_active":"2024-04-26T19:50:30.109147+0000","last_became_peered":"2024-04-26T19:50:30.109147+0000","last_unstale":"2024-04-26T19:50:38.653084+0000","last_undegraded":"2024-04-26T19:50:38.653084+0000","last_fullsized":"2024-04-26T19:50:38.653084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654171+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:50:38.654171+0000","last_peered":"2024-04-26T19:50:38.654171+0000","last_clean":"2024-04-26T19:50:38.654171+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:50:38.654171+0000","last_undegraded":"2024-04-26T19:50:38.654171+0000","last_fullsized":"2024-04-26T19:50:38.654171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653046+0000","last_change":"2024-04-26T19:50:35.627345+0000","last_active":"2024-04-26T19:50:38.653046+0000","last_peered":"2024-04-26T19:50:38.653046+0000","last_clean":"2024-04-26T19:50:38.653046+0000","last_became_active":"2024-04-26T19:50:35.626795+0000","last_became_peered":"2024-04-26T19:50:35.626795+0000","last_unstale":"2024-04-26T19:50:38.653046+0000","last_undegraded":"2024-04-26T19:50:38.653046+0000","last_fullsized":"2024-04-26T19:50:38.653046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:14.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653270+0000","last_change":"2024-04-26T19:50:31.112950+0000","last_active":"2024-04-26T19:50:38.653270+0000","last_peered":"2024-04-26T19:50:38.653270+0000","last_clean":"2024-04-26T19:50:38.653270+0000","last_became_active":"2024-04-26T19:50:31.112674+0000","last_became_peered":"2024-04-26T19:50:31.112674+0000","last_unstale":"2024-04-26T19:50:38.653270+0000","last_undegraded":"2024-04-26T19:50:38.653270+0000","last_fullsized":"2024-04-26T19:50:38.653270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537253+0000","last_change":"2024-04-26T19:50:30.114819+0000","last_active":"2024-04-26T19:50:38.537253+0000","last_peered":"2024-04-26T19:50:38.537253+0000","last_clean":"2024-04-26T19:50:38.537253+0000","last_became_active":"2024-04-26T19:50:30.114566+0000","last_became_peered":"2024-04-26T19:50:30.114566+0000","last_unstale":"2024-04-26T19:50:38.537253+0000","last_undegraded":"2024-04-26T19:50:38.537253+0000","last_fullsized":"2024-04-26T19:50:38.537253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654387+0000","last_change":"2024-04-26T19:50:35.626488+0000","last_active":"2024-04-26T19:50:38.654387+0000","last_peered":"2024-04-26T19:50:38.654387+0000","last_clean":"2024-04-26T19:50:38.654387+0000","last_became_active":"2024-04-26T19:50:35.625951+0000","last_became_peered":"2024-04-26T19:50:35.625951+0000","last_unstale":"2024-04-26T19:50:38.654387+0000","last_undegraded":"2024-04-26T19:50:38.654387+0000","last_fullsized":"2024-04-26T19:50:38.654387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:46.591736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652684+0000","last_change":"2024-04-26T19:50:31.119380+0000","last_active":"2024-04-26T19:50:38.652684+0000","last_peered":"2024-04-26T19:50:38.652684+0000","last_clean":"2024-04-26T19:50:38.652684+0000","last_became_active":"2024-04-26T19:50:31.118011+0000","last_became_peered":"2024-04-26T19:50:31.118011+0000","last_unstale":"2024-04-26T19:50:38.652684+0000","last_undegraded":"2024-04-26T19:50:38.652684+0000","last_fullsized":"2024-04-26T19:50:38.652684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:13.577254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997787+0000","last_change":"2024-04-26T19:50:30.101571+0000","last_active":"2024-04-26T19:50:37.997787+0000","last_peered":"2024-04-26T19:50:37.997787+0000","last_clean":"2024-04-26T19:50:37.997787+0000","last_became_active":"2024-04-26T19:50:30.101257+0000","last_became_peered":"2024-04-26T19:50:30.101257+0000","last_unstale":"2024-04-26T19:50:37.997787+0000","last_undegraded":"2024-04-26T19:50:37.997787+0000","last_fullsized":"2024-04-26T19:50:37.997787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:13.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299968+0000","last_change":"2024-04-26T19:50:35.146304+0000","last_active":"2024-04-26T19:50:38.299968+0000","last_peered":"2024-04-26T19:50:38.299968+0000","last_clean":"2024-04-26T19:50:38.299968+0000","last_became_active":"2024-04-26T19:50:35.145918+0000","last_became_peered":"2024-04-26T19:50:35.145918+0000","last_unstale":"2024-04-26T19:50:38.299968+0000","last_undegraded":"2024-04-26T19:50:38.299968+0000","last_fullsized":"2024-04-26T19:50:38.299968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653338+0000","last_change":"2024-04-26T19:50:31.115622+0000","last_active":"2024-04-26T19:50:38.653338+0000","last_peered":"2024-04-26T19:50:38.653338+0000","last_clean":"2024-04-26T19:50:38.653338+0000","last_became_active":"2024-04-26T19:50:31.115358+0000","last_became_peered":"2024-04-26T19:50:31.115358+0000","last_unstale":"2024-04-26T19:50:38.653338+0000","last_undegraded":"2024-04-26T19:50:38.653338+0000","last_fullsized":"2024-04-26T19:50:38.653338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:31.186700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536869+0000","last_change":"2024-04-26T19:50:30.112087+0000","last_active":"2024-04-26T19:50:38.536869+0000","last_peered":"2024-04-26T19:50:38.536869+0000","last_clean":"2024-04-26T19:50:38.536869+0000","last_became_active":"2024-04-26T19:50:30.111773+0000","last_became_peered":"2024-04-26T19:50:30.111773+0000","last_unstale":"2024-04-26T19:50:38.536869+0000","last_undegraded":"2024-04-26T19:50:38.536869+0000","last_fullsized":"2024-04-26T19:50:38.536869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652160+0000","last_change":"2024-04-26T19:50:35.624170+0000","last_active":"2024-04-26T19:50:38.652160+0000","last_peered":"2024-04-26T19:50:38.652160+0000","last_clean":"2024-04-26T19:50:38.652160+0000","last_became_active":"2024-04-26T19:50:35.623758+0000","last_became_peered":"2024-04-26T19:50:35.623758+0000","last_unstale":"2024-04-26T19:50:38.652160+0000","last_undegraded":"2024-04-26T19:50:38.652160+0000","last_fullsized":"2024-04-26T19:50:38.652160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:01.350601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653614+0000","last_change":"2024-04-26T19:50:31.115100+0000","last_active":"2024-04-26T19:50:38.653614+0000","last_peered":"2024-04-26T19:50:38.653614+0000","last_clean":"2024-04-26T19:50:38.653614+0000","last_became_active":"2024-04-26T19:50:31.114558+0000","last_became_peered":"2024-04-26T19:50:31.114558+0000","last_unstale":"2024-04-26T19:50:38.653614+0000","last_undegraded":"2024-04-26T19:50:38.653614+0000","last_fullsized":"2024-04-26T19:50:38.653614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:42.558104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130407+0000","last_change":"2024-04-26T19:50:30.100580+0000","last_active":"2024-04-26T19:50:37.130407+0000","last_peered":"2024-04-26T19:50:37.130407+0000","last_clean":"2024-04-26T19:50:37.130407+0000","last_became_active":"2024-04-26T19:50:30.099562+0000","last_became_peered":"2024-04-26T19:50:30.099562+0000","last_unstale":"2024-04-26T19:50:37.130407+0000","last_undegraded":"2024-04-26T19:50:37.130407+0000","last_fullsized":"2024-04-26T19:50:37.130407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:02.207190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129529+0000","last_change":"2024-04-26T19:50:35.143446+0000","last_active":"2024-04-26T19:50:37.129529+0000","last_peered":"2024-04-26T19:50:37.129529+0000","last_clean":"2024-04-26T19:50:37.129529+0000","last_became_active":"2024-04-26T19:50:35.143028+0000","last_became_peered":"2024-04-26T19:50:35.143028+0000","last_unstale":"2024-04-26T19:50:37.129529+0000","last_undegraded":"2024-04-26T19:50:37.129529+0000","last_fullsized":"2024-04-26T19:50:37.129529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:43.974896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653088+0000","last_change":"2024-04-26T19:50:31.116546+0000","last_active":"2024-04-26T19:50:38.653088+0000","last_peered":"2024-04-26T19:50:38.653088+0000","last_clean":"2024-04-26T19:50:38.653088+0000","last_became_active":"2024-04-26T19:50:31.115261+0000","last_became_peered":"2024-04-26T19:50:31.115261+0000","last_unstale":"2024-04-26T19:50:38.653088+0000","last_undegraded":"2024-04-26T19:50:38.653088+0000","last_fullsized":"2024-04-26T19:50:38.653088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.212877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536837+0000","last_change":"2024-04-26T19:50:30.099768+0000","last_active":"2024-04-26T19:50:38.536837+0000","last_peered":"2024-04-26T19:50:38.536837+0000","last_clean":"2024-04-26T19:50:38.536837+0000","last_became_active":"2024-04-26T19:50:30.099471+0000","last_became_peered":"2024-04-26T19:50:30.099471+0000","last_unstale":"2024-04-26T19:50:38.536837+0000","last_undegraded":"2024-04-26T19:50:38.536837+0000","last_fullsized":"2024-04-26T19:50:38.536837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299923+0000","last_change":"2024-04-26T19:50:35.148077+0000","last_active":"2024-04-26T19:50:38.299923+0000","last_peered":"2024-04-26T19:50:38.299923+0000","last_clean":"2024-04-26T19:50:38.299923+0000","last_became_active":"2024-04-26T19:50:35.147552+0000","last_became_peered":"2024-04-26T19:50:35.147552+0000","last_unstale":"2024-04-26T19:50:38.299923+0000","last_undegraded":"2024-04-26T19:50:38.299923+0000","last_fullsized":"2024-04-26T19:50:38.299923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:11.052818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652600+0000","last_change":"2024-04-26T19:50:31.105398+0000","last_active":"2024-04-26T19:50:38.652600+0000","last_peered":"2024-04-26T19:50:38.652600+0000","last_clean":"2024-04-26T19:50:38.652600+0000","last_became_active":"2024-04-26T19:50:31.105140+0000","last_became_peered":"2024-04-26T19:50:31.105140+0000","last_unstale":"2024-04-26T19:50:38.652600+0000","last_undegraded":"2024-04-26T19:50:38.652600+0000","last_fullsized":"2024-04-26T19:50:38.652600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.651966+0000","last_change":"2024-04-26T19:50:30.111019+0000","last_active":"2024-04-26T19:50:38.651966+0000","last_peered":"2024-04-26T19:50:38.651966+0000","last_clean":"2024-04-26T19:50:38.651966+0000","last_became_active":"2024-04-26T19:50:30.110590+0000","last_became_peered":"2024-04-26T19:50:30.110590+0000","last_unstale":"2024-04-26T19:50:38.651966+0000","last_undegraded":"2024-04-26T19:50:38.651966+0000","last_fullsized":"2024-04-26T19:50:38.651966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131001+0000","last_change":"2024-04-26T19:50:35.625721+0000","last_active":"2024-04-26T19:50:37.131001+0000","last_peered":"2024-04-26T19:50:37.131001+0000","last_clean":"2024-04-26T19:50:37.131001+0000","last_became_active":"2024-04-26T19:50:35.625294+0000","last_became_peered":"2024-04-26T19:50:35.625294+0000","last_unstale":"2024-04-26T19:50:37.131001+0000","last_undegraded":"2024-04-26T19:50:37.131001+0000","last_fullsized":"2024-04-26T19:50:37.131001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:18.158960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653765+0000","last_change":"2024-04-26T19:50:31.113984+0000","last_active":"2024-04-26T19:50:38.653765+0000","last_peered":"2024-04-26T19:50:38.653765+0000","last_clean":"2024-04-26T19:50:38.653765+0000","last_became_active":"2024-04-26T19:50:31.113723+0000","last_became_peered":"2024-04-26T19:50:31.113723+0000","last_unstale":"2024-04-26T19:50:38.653765+0000","last_undegraded":"2024-04-26T19:50:38.653765+0000","last_fullsized":"2024-04-26T19:50:38.653765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653698+0000","last_change":"2024-04-26T19:50:30.110687+0000","last_active":"2024-04-26T19:50:38.653698+0000","last_peered":"2024-04-26T19:50:38.653698+0000","last_clean":"2024-04-26T19:50:38.653698+0000","last_became_active":"2024-04-26T19:50:30.110423+0000","last_became_peered":"2024-04-26T19:50:30.110423+0000","last_unstale":"2024-04-26T19:50:38.653698+0000","last_undegraded":"2024-04-26T19:50:38.653698+0000","last_fullsized":"2024-04-26T19:50:38.653698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130741+0000","last_change":"2024-04-26T19:50:35.623855+0000","last_active":"2024-04-26T19:50:37.130741+0000","last_peered":"2024-04-26T19:50:37.130741+0000","last_clean":"2024-04-26T19:50:37.130741+0000","last_became_active":"2024-04-26T19:50:35.623453+0000","last_became_peered":"2024-04-26T19:50:35.623453+0000","last_unstale":"2024-04-26T19:50:37.130741+0000","last_undegraded":"2024-04-26T19:50:37.130741+0000","last_fullsized":"2024-04-26T19:50:37.130741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:21.199810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299726+0000","last_change":"2024-04-26T19:50:31.114891+0000","last_active":"2024-04-26T19:50:38.299726+0000","last_peered":"2024-04-26T19:50:38.299726+0000","last_clean":"2024-04-26T19:50:38.299726+0000","last_became_active":"2024-04-26T19:50:31.114338+0000","last_became_peered":"2024-04-26T19:50:31.114338+0000","last_unstale":"2024-04-26T19:50:38.299726+0000","last_undegraded":"2024-04-26T19:50:38.299726+0000","last_fullsized":"2024-04-26T19:50:38.299726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299301+0000","last_change":"2024-04-26T19:50:30.108408+0000","last_active":"2024-04-26T19:50:38.299301+0000","last_peered":"2024-04-26T19:50:38.299301+0000","last_clean":"2024-04-26T19:50:38.299301+0000","last_became_active":"2024-04-26T19:50:30.107261+0000","last_became_peered":"2024-04-26T19:50:30.107261+0000","last_unstale":"2024-04-26T19:50:38.299301+0000","last_undegraded":"2024-04-26T19:50:38.299301+0000","last_fullsized":"2024-04-26T19:50:38.299301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:21.153340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654534+0000","last_change":"2024-04-26T19:50:35.624403+0000","last_active":"2024-04-26T19:50:38.654534+0000","last_peered":"2024-04-26T19:50:38.654534+0000","last_clean":"2024-04-26T19:50:38.654534+0000","last_became_active":"2024-04-26T19:50:35.623811+0000","last_became_peered":"2024-04-26T19:50:35.623811+0000","last_unstale":"2024-04-26T19:50:38.654534+0000","last_undegraded":"2024-04-26T19:50:38.654534+0000","last_fullsized":"2024-04-26T19:50:38.654534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.747785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653171+0000","last_change":"2024-04-26T19:50:31.115452+0000","last_active":"2024-04-26T19:50:38.653171+0000","last_peered":"2024-04-26T19:50:38.653171+0000","last_clean":"2024-04-26T19:50:38.653171+0000","last_became_active":"2024-04-26T19:50:31.114780+0000","last_became_peered":"2024-04-26T19:50:31.114780+0000","last_unstale":"2024-04-26T19:50:38.653171+0000","last_undegraded":"2024-04-26T19:50:38.653171+0000","last_fullsized":"2024-04-26T19:50:38.653171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:40.502106+0000","last_change":"2024-04-26T19:50:30.110540+0000","last_active":"2024-04-26T19:50:40.502106+0000","last_peered":"2024-04-26T19:50:40.502106+0000","last_clean":"2024-04-26T19:50:40.502106+0000","last_became_active":"2024-04-26T19:50:30.110241+0000","last_became_peered":"2024-04-26T19:50:30.110241+0000","last_unstale":"2024-04-26T19:50:40.502106+0000","last_undegraded":"2024-04-26T19:50:40.502106+0000","last_fullsized":"2024-04-26T19:50:40.502106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997615+0000","last_change":"2024-04-26T19:50:35.149139+0000","last_active":"2024-04-26T19:50:37.997615+0000","last_peered":"2024-04-26T19:50:37.997615+0000","last_clean":"2024-04-26T19:50:37.997615+0000","last_became_active":"2024-04-26T19:50:35.148663+0000","last_became_peered":"2024-04-26T19:50:35.148663+0000","last_unstale":"2024-04-26T19:50:37.997615+0000","last_undegraded":"2024-04-26T19:50:37.997615+0000","last_fullsized":"2024-04-26T19:50:37.997615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130224+0000","last_change":"2024-04-26T19:50:31.112921+0000","last_active":"2024-04-26T19:50:37.130224+0000","last_peered":"2024-04-26T19:50:37.130224+0000","last_clean":"2024-04-26T19:50:37.130224+0000","last_became_active":"2024-04-26T19:50:31.112651+0000","last_became_peered":"2024-04-26T19:50:31.112651+0000","last_unstale":"2024-04-26T19:50:37.130224+0000","last_undegraded":"2024-04-26T19:50:37.130224+0000","last_fullsized":"2024-04-26T19:50:37.130224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:13.149763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997777+0000","last_change":"2024-04-26T19:50:30.110464+0000","last_active":"2024-04-26T19:50:37.997777+0000","last_peered":"2024-04-26T19:50:37.997777+0000","last_clean":"2024-04-26T19:50:37.997777+0000","last_became_active":"2024-04-26T19:50:30.110101+0000","last_became_peered":"2024-04-26T19:50:30.110101+0000","last_unstale":"2024-04-26T19:50:37.997777+0000","last_undegraded":"2024-04-26T19:50:37.997777+0000","last_fullsized":"2024-04-26T19:50:37.997777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:12.144175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130067+0000","last_change":"2024-04-26T19:50:35.143854+0000","last_active":"2024-04-26T19:50:37.130067+0000","last_peered":"2024-04-26T19:50:37.130067+0000","last_clean":"2024-04-26T19:50:37.130067+0000","last_became_active":"2024-04-26T19:50:35.143264+0000","last_became_peered":"2024-04-26T19:50:35.143264+0000","last_unstale":"2024-04-26T19:50:37.130067+0000","last_undegraded":"2024-04-26T19:50:37.130067+0000","last_fullsized":"2024-04-26T19:50:37.130067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300209+0000","last_change":"2024-04-26T19:50:31.115638+0000","last_active":"2024-04-26T19:50:38.300209+0000","last_peered":"2024-04-26T19:50:38.300209+0000","last_clean":"2024-04-26T19:50:38.300209+0000","last_became_active":"2024-04-26T19:50:31.113889+0000","last_became_peered":"2024-04-26T19:50:31.113889+0000","last_unstale":"2024-04-26T19:50:38.300209+0000","last_undegraded":"2024-04-26T19:50:38.300209+0000","last_fullsized":"2024-04-26T19:50:38.300209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652958+0000","last_change":"2024-04-26T19:50:30.109848+0000","last_active":"2024-04-26T19:50:38.652958+0000","last_peered":"2024-04-26T19:50:38.652958+0000","last_clean":"2024-04-26T19:50:38.652958+0000","last_became_active":"2024-04-26T19:50:30.109407+0000","last_became_peered":"2024-04-26T19:50:30.109407+0000","last_unstale":"2024-04-26T19:50:38.652958+0000","last_undegraded":"2024-04-26T19:50:38.652958+0000","last_fullsized":"2024-04-26T19:50:38.652958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130343+0000","last_change":"2024-04-26T19:50:35.143221+0000","last_active":"2024-04-26T19:50:37.130343+0000","last_peered":"2024-04-26T19:50:37.130343+0000","last_clean":"2024-04-26T19:50:37.130343+0000","last_became_active":"2024-04-26T19:50:35.142868+0000","last_became_peered":"2024-04-26T19:50:35.142868+0000","last_unstale":"2024-04-26T19:50:37.130343+0000","last_undegraded":"2024-04-26T19:50:37.130343+0000","last_fullsized":"2024-04-26T19:50:37.130343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:16.097115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652135+0000","last_change":"2024-04-26T19:50:31.115770+0000","last_active":"2024-04-26T19:50:38.652135+0000","last_peered":"2024-04-26T19:50:38.652135+0000","last_clean":"2024-04-26T19:50:38.652135+0000","last_became_active":"2024-04-26T19:50:31.115462+0000","last_became_peered":"2024-04-26T19:50:31.115462+0000","last_unstale":"2024-04-26T19:50:38.652135+0000","last_undegraded":"2024-04-26T19:50:38.652135+0000","last_fullsized":"2024-04-26T19:50:38.652135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.212877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653880+0000","last_change":"2024-04-26T19:50:30.110213+0000","last_active":"2024-04-26T19:50:38.653880+0000","last_peered":"2024-04-26T19:50:38.653880+0000","last_clean":"2024-04-26T19:50:38.653880+0000","last_became_active":"2024-04-26T19:50:30.109923+0000","last_became_peered":"2024-04-26T19:50:30.109923+0000","last_unstale":"2024-04-26T19:50:38.653880+0000","last_undegraded":"2024-04-26T19:50:38.653880+0000","last_fullsized":"2024-04-26T19:50:38.653880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130324+0000","last_change":"2024-04-26T19:50:35.624637+0000","last_active":"2024-04-26T19:50:37.130324+0000","last_peered":"2024-04-26T19:50:37.130324+0000","last_clean":"2024-04-26T19:50:37.130324+0000","last_became_active":"2024-04-26T19:50:35.624195+0000","last_became_peered":"2024-04-26T19:50:35.624195+0000","last_unstale":"2024-04-26T19:50:37.130324+0000","last_undegraded":"2024-04-26T19:50:37.130324+0000","last_fullsized":"2024-04-26T19:50:37.130324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:16.049053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299925+0000","last_change":"2024-04-26T19:50:35.626111+0000","last_active":"2024-04-26T19:50:38.299925+0000","last_peered":"2024-04-26T19:50:38.299925+0000","last_clean":"2024-04-26T19:50:38.299925+0000","last_became_active":"2024-04-26T19:50:35.625678+0000","last_became_peered":"2024-04-26T19:50:35.625678+0000","last_unstale":"2024-04-26T19:50:38.299925+0000","last_undegraded":"2024-04-26T19:50:38.299925+0000","last_fullsized":"2024-04-26T19:50:38.299925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997291+0000","last_change":"2024-04-26T19:50:31.108163+0000","last_active":"2024-04-26T19:50:37.997291+0000","last_peered":"2024-04-26T19:50:37.997291+0000","last_clean":"2024-04-26T19:50:37.997291+0000","last_became_active":"2024-04-26T19:50:31.107840+0000","last_became_peered":"2024-04-26T19:50:31.107840+0000","last_unstale":"2024-04-26T19:50:37.997291+0000","last_undegraded":"2024-04-26T19:50:37.997291+0000","last_fullsized":"2024-04-26T19:50:37.997291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.651925+0000","last_change":"2024-04-26T19:50:30.111205+0000","last_active":"2024-04-26T19:50:38.651925+0000","last_peered":"2024-04-26T19:50:38.651925+0000","last_clean":"2024-04-26T19:50:38.651925+0000","last_became_active":"2024-04-26T19:50:30.110491+0000","last_became_peered":"2024-04-26T19:50:30.110491+0000","last_unstale":"2024-04-26T19:50:38.651925+0000","last_undegraded":"2024-04-26T19:50:38.651925+0000","last_fullsized":"2024-04-26T19:50:38.651925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653880+0000","last_change":"2024-04-26T19:50:35.144010+0000","last_active":"2024-04-26T19:50:38.653880+0000","last_peered":"2024-04-26T19:50:38.653880+0000","last_clean":"2024-04-26T19:50:38.653880+0000","last_became_active":"2024-04-26T19:50:35.143558+0000","last_became_peered":"2024-04-26T19:50:35.143558+0000","last_unstale":"2024-04-26T19:50:38.653880+0000","last_undegraded":"2024-04-26T19:50:38.653880+0000","last_fullsized":"2024-04-26T19:50:38.653880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:10.325286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130338+0000","last_change":"2024-04-26T19:50:31.106592+0000","last_active":"2024-04-26T19:50:37.130338+0000","last_peered":"2024-04-26T19:50:37.130338+0000","last_clean":"2024-04-26T19:50:37.130338+0000","last_became_active":"2024-04-26T19:50:31.106298+0000","last_became_peered":"2024-04-26T19:50:31.106298+0000","last_unstale":"2024-04-26T19:50:37.130338+0000","last_undegraded":"2024-04-26T19:50:37.130338+0000","last_fullsized":"2024-04-26T19:50:37.130338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130435+0000","last_change":"2024-04-26T19:50:30.110713+0000","last_active":"2024-04-26T19:50:37.130435+0000","last_peered":"2024-04-26T19:50:37.130435+0000","last_clean":"2024-04-26T19:50:37.130435+0000","last_became_active":"2024-04-26T19:50:30.109274+0000","last_became_peered":"2024-04-26T19:50:30.109274+0000","last_unstale":"2024-04-26T19:50:37.130435+0000","last_undegraded":"2024-04-26T19:50:37.130435+0000","last_fullsized":"2024-04-26T19:50:37.130435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654164+0000","last_change":"2024-04-26T19:50:30.111443+0000","last_active":"2024-04-26T19:50:38.654164+0000","last_peered":"2024-04-26T19:50:38.654164+0000","last_clean":"2024-04-26T19:50:38.654164+0000","last_became_active":"2024-04-26T19:50:30.111159+0000","last_became_peered":"2024-04-26T19:50:30.111159+0000","last_unstale":"2024-04-26T19:50:38.654164+0000","last_undegraded":"2024-04-26T19:50:38.654164+0000","last_fullsized":"2024-04-26T19:50:38.654164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300079+0000","last_change":"2024-04-26T19:50:31.104770+0000","last_active":"2024-04-26T19:50:38.300079+0000","last_peered":"2024-04-26T19:50:38.300079+0000","last_clean":"2024-04-26T19:50:38.300079+0000","last_became_active":"2024-04-26T19:50:31.104100+0000","last_became_peered":"2024-04-26T19:50:31.104100+0000","last_unstale":"2024-04-26T19:50:38.300079+0000","last_undegraded":"2024-04-26T19:50:38.300079+0000","last_fullsized":"2024-04-26T19:50:38.300079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537176+0000","last_change":"2024-04-26T19:50:35.143179+0000","last_active":"2024-04-26T19:50:38.537176+0000","last_peered":"2024-04-26T19:50:38.537176+0000","last_clean":"2024-04-26T19:50:38.537176+0000","last_became_active":"2024-04-26T19:50:35.142761+0000","last_became_peered":"2024-04-26T19:50:35.142761+0000","last_unstale":"2024-04-26T19:50:38.537176+0000","last_undegraded":"2024-04-26T19:50:38.537176+0000","last_fullsized":"2024-04-26T19:50:38.537176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:42.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130613+0000","last_change":"2024-04-26T19:50:30.115508+0000","last_active":"2024-04-26T19:50:37.130613+0000","last_peered":"2024-04-26T19:50:37.130613+0000","last_clean":"2024-04-26T19:50:37.130613+0000","last_became_active":"2024-04-26T19:50:30.115235+0000","last_became_peered":"2024-04-26T19:50:30.115235+0000","last_unstale":"2024-04-26T19:50:37.130613+0000","last_undegraded":"2024-04-26T19:50:37.130613+0000","last_fullsized":"2024-04-26T19:50:37.130613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:31.691383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129914+0000","last_change":"2024-04-26T19:50:31.110879+0000","last_active":"2024-04-26T19:50:37.129914+0000","last_peered":"2024-04-26T19:50:37.129914+0000","last_clean":"2024-04-26T19:50:37.129914+0000","last_became_active":"2024-04-26T19:50:31.110592+0000","last_became_peered":"2024-04-26T19:50:31.110592+0000","last_unstale":"2024-04-26T19:50:37.129914+0000","last_undegraded":"2024-04-26T19:50:37.129914+0000","last_fullsized":"2024-04-26T19:50:37.129914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:54.410618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130887+0000","last_change":"2024-04-26T19:50:35.144257+0000","last_active":"2024-04-26T19:50:37.130887+0000","last_peered":"2024-04-26T19:50:37.130887+0000","last_clean":"2024-04-26T19:50:37.130887+0000","last_became_active":"2024-04-26T19:50:35.143853+0000","last_became_peered":"2024-04-26T19:50:35.143853+0000","last_unstale":"2024-04-26T19:50:37.130887+0000","last_undegraded":"2024-04-26T19:50:37.130887+0000","last_fullsized":"2024-04-26T19:50:37.130887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:05.350339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652235+0000","last_change":"2024-04-26T19:50:31.105355+0000","last_active":"2024-04-26T19:50:38.652235+0000","last_peered":"2024-04-26T19:50:38.652235+0000","last_clean":"2024-04-26T19:50:38.652235+0000","last_became_active":"2024-04-26T19:50:31.105085+0000","last_became_peered":"2024-04-26T19:50:31.105085+0000","last_unstale":"2024-04-26T19:50:38.652235+0000","last_undegraded":"2024-04-26T19:50:38.652235+0000","last_fullsized":"2024-04-26T19:50:38.652235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536962+0000","last_change":"2024-04-26T19:50:31.112415+0000","last_active":"2024-04-26T19:50:38.536962+0000","last_peered":"2024-04-26T19:50:38.536962+0000","last_clean":"2024-04-26T19:50:38.536962+0000","last_became_active":"2024-04-26T19:50:31.112126+0000","last_became_peered":"2024-04-26T19:50:31.112126+0000","last_unstale":"2024-04-26T19:50:38.536962+0000","last_undegraded":"2024-04-26T19:50:38.536962+0000","last_fullsized":"2024-04-26T19:50:38.536962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:13.149763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654244+0000","last_change":"2024-04-26T19:50:31.121580+0000","last_active":"2024-04-26T19:50:38.654244+0000","last_peered":"2024-04-26T19:50:38.654244+0000","last_clean":"2024-04-26T19:50:38.654244+0000","last_became_active":"2024-04-26T19:50:31.121296+0000","last_became_peered":"2024-04-26T19:50:31.121296+0000","last_unstale":"2024-04-26T19:50:38.654244+0000","last_undegraded":"2024-04-26T19:50:38.654244+0000","last_fullsized":"2024-04-26T19:50:38.654244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.291653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300008+0000","last_change":"2024-04-26T19:50:31.106600+0000","last_active":"2024-04-26T19:50:38.300008+0000","last_peered":"2024-04-26T19:50:38.300008+0000","last_clean":"2024-04-26T19:50:38.300008+0000","last_became_active":"2024-04-26T19:50:31.106120+0000","last_became_peered":"2024-04-26T19:50:31.106120+0000","last_unstale":"2024-04-26T19:50:38.300008+0000","last_undegraded":"2024-04-26T19:50:38.300008+0000","last_fullsized":"2024-04-26T19:50:38.300008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130813+0000","last_change":"2024-04-26T19:50:31.113865+0000","last_active":"2024-04-26T19:50:37.130813+0000","last_peered":"2024-04-26T19:50:37.130813+0000","last_clean":"2024-04-26T19:50:37.130813+0000","last_became_active":"2024-04-26T19:50:31.113603+0000","last_became_peered":"2024-04-26T19:50:31.113603+0000","last_unstale":"2024-04-26T19:50:37.130813+0000","last_undegraded":"2024-04-26T19:50:37.130813+0000","last_fullsized":"2024-04-26T19:50:37.130813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997661+0000","last_change":"2024-04-26T19:50:31.113048+0000","last_active":"2024-04-26T19:50:37.997661+0000","last_peered":"2024-04-26T19:50:37.997661+0000","last_clean":"2024-04-26T19:50:37.997661+0000","last_became_active":"2024-04-26T19:50:31.112760+0000","last_became_peered":"2024-04-26T19:50:31.112760+0000","last_unstale":"2024-04-26T19:50:37.997661+0000","last_undegraded":"2024-04-26T19:50:37.997661+0000","last_fullsized":"2024-04-26T19:50:37.997661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653673+0000","last_change":"2024-04-26T19:50:31.116289+0000","last_active":"2024-04-26T19:50:38.653673+0000","last_peered":"2024-04-26T19:50:38.653673+0000","last_clean":"2024-04-26T19:50:38.653673+0000","last_became_active":"2024-04-26T19:50:31.116030+0000","last_became_peered":"2024-04-26T19:50:31.116030+0000","last_unstale":"2024-04-26T19:50:38.653673+0000","last_undegraded":"2024-04-26T19:50:38.653673+0000","last_fullsized":"2024-04-26T19:50:38.653673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130946+0000","last_change":"2024-04-26T19:50:31.106460+0000","last_active":"2024-04-26T19:50:37.130946+0000","last_peered":"2024-04-26T19:50:37.130946+0000","last_clean":"2024-04-26T19:50:37.130946+0000","last_became_active":"2024-04-26T19:50:31.106066+0000","last_became_peered":"2024-04-26T19:50:31.106066+0000","last_unstale":"2024-04-26T19:50:37.130946+0000","last_undegraded":"2024-04-26T19:50:37.130946+0000","last_fullsized":"2024-04-26T19:50:37.130946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652449+0000","last_change":"2024-04-26T19:50:31.121061+0000","last_active":"2024-04-26T19:50:38.652449+0000","last_peered":"2024-04-26T19:50:38.652449+0000","last_clean":"2024-04-26T19:50:38.652449+0000","last_became_active":"2024-04-26T19:50:31.120795+0000","last_became_peered":"2024-04-26T19:50:31.120795+0000","last_unstale":"2024-04-26T19:50:38.652449+0000","last_undegraded":"2024-04-26T19:50:38.652449+0000","last_fullsized":"2024-04-26T19:50:38.652449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130353+0000","last_change":"2024-04-26T19:50:31.108437+0000","last_active":"2024-04-26T19:50:37.130353+0000","last_peered":"2024-04-26T19:50:37.130353+0000","last_clean":"2024-04-26T19:50:37.130353+0000","last_became_active":"2024-04-26T19:50:31.108163+0000","last_became_peered":"2024-04-26T19:50:31.108163+0000","last_unstale":"2024-04-26T19:50:37.130353+0000","last_undegraded":"2024-04-26T19:50:37.130353+0000","last_fullsized":"2024-04-26T19:50:37.130353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654157+0000","last_change":"2024-04-26T19:50:31.117685+0000","last_active":"2024-04-26T19:50:38.654157+0000","last_peered":"2024-04-26T19:50:38.654157+0000","last_clean":"2024-04-26T19:50:38.654157+0000","last_became_active":"2024-04-26T19:50:31.115588+0000","last_became_peered":"2024-04-26T19:50:31.115588+0000","last_unstale":"2024-04-26T19:50:38.654157+0000","last_undegraded":"2024-04-26T19:50:38.654157+0000","last_fullsized":"2024-04-26T19:50:38.654157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:10.399495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299776+0000","last_change":"2024-04-26T19:50:31.115273+0000","last_active":"2024-04-26T19:50:38.299776+0000","last_peered":"2024-04-26T19:50:38.299776+0000","last_clean":"2024-04-26T19:50:38.299776+0000","last_became_active":"2024-04-26T19:50:31.114772+0000","last_became_peered":"2024-04-26T19:50:31.114772+0000","last_unstale":"2024-04-26T19:50:38.299776+0000","last_undegraded":"2024-04-26T19:50:38.299776+0000","last_fullsized":"2024-04-26T19:50:38.299776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653493+0000","last_change":"2024-04-26T19:50:31.110492+0000","last_active":"2024-04-26T19:50:38.653493+0000","last_peered":"2024-04-26T19:50:38.653493+0000","last_clean":"2024-04-26T19:50:38.653493+0000","last_became_active":"2024-04-26T19:50:31.110280+0000","last_became_peered":"2024-04-26T19:50:31.110280+0000","last_unstale":"2024-04-26T19:50:38.653493+0000","last_undegraded":"2024-04-26T19:50:38.653493+0000","last_fullsized":"2024-04-26T19:50:38.653493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653886+0000","last_change":"2024-04-26T19:50:31.114136+0000","last_active":"2024-04-26T19:50:38.653886+0000","last_peered":"2024-04-26T19:50:38.653886+0000","last_clean":"2024-04-26T19:50:38.653886+0000","last_became_active":"2024-04-26T19:50:31.113175+0000","last_became_peered":"2024-04-26T19:50:31.113175+0000","last_unstale":"2024-04-26T19:50:38.653886+0000","last_undegraded":"2024-04-26T19:50:38.653886+0000","last_fullsized":"2024-04-26T19:50:38.653886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:39.941264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537153+0000","last_change":"2024-04-26T19:50:31.108339+0000","last_active":"2024-04-26T19:50:38.537153+0000","last_peered":"2024-04-26T19:50:38.537153+0000","last_clean":"2024-04-26T19:50:38.537153+0000","last_became_active":"2024-04-26T19:50:31.108092+0000","last_became_peered":"2024-04-26T19:50:31.108092+0000","last_unstale":"2024-04-26T19:50:38.537153+0000","last_undegraded":"2024-04-26T19:50:38.537153+0000","last_fullsized":"2024-04-26T19:50:38.537153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:14.980411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653311+0000","last_change":"2024-04-26T19:50:31.116207+0000","last_active":"2024-04-26T19:50:38.653311+0000","last_peered":"2024-04-26T19:50:38.653311+0000","last_clean":"2024-04-26T19:50:38.653311+0000","last_became_active":"2024-04-26T19:50:31.115498+0000","last_became_peered":"2024-04-26T19:50:31.115498+0000","last_unstale":"2024-04-26T19:50:38.653311+0000","last_undegraded":"2024-04-26T19:50:38.653311+0000","last_fullsized":"2024-04-26T19:50:38.653311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:42.558104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652346+0000","last_change":"2024-04-26T19:50:31.117352+0000","last_active":"2024-04-26T19:50:38.652346+0000","last_peered":"2024-04-26T19:50:38.652346+0000","last_clean":"2024-04-26T19:50:38.652346+0000","last_became_active":"2024-04-26T19:50:31.117098+0000","last_became_peered":"2024-04-26T19:50:31.117098+0000","last_unstale":"2024-04-26T19:50:38.652346+0000","last_undegraded":"2024-04-26T19:50:38.652346+0000","last_fullsized":"2024-04-26T19:50:38.652346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997376+0000","last_change":"2024-04-26T19:50:31.113884+0000","last_active":"2024-04-26T19:50:37.997376+0000","last_peered":"2024-04-26T19:50:37.997376+0000","last_clean":"2024-04-26T19:50:37.997376+0000","last_became_active":"2024-04-26T19:50:31.113607+0000","last_became_peered":"2024-04-26T19:50:31.113607+0000","last_unstale":"2024-04-26T19:50:37.997376+0000","last_undegraded":"2024-04-26T19:50:37.997376+0000","last_fullsized":"2024-04-26T19:50:37.997376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299710+0000","last_change":"2024-04-26T19:50:31.110086+0000","last_active":"2024-04-26T19:50:38.299710+0000","last_peered":"2024-04-26T19:50:38.299710+0000","last_clean":"2024-04-26T19:50:38.299710+0000","last_became_active":"2024-04-26T19:50:31.109800+0000","last_became_peered":"2024-04-26T19:50:31.109800+0000","last_unstale":"2024-04-26T19:50:38.299710+0000","last_undegraded":"2024-04-26T19:50:38.299710+0000","last_fullsized":"2024-04-26T19:50:38.299710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:45.295846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130514+0000","last_change":"2024-04-26T19:50:31.109078+0000","last_active":"2024-04-26T19:50:37.130514+0000","last_peered":"2024-04-26T19:50:37.130514+0000","last_clean":"2024-04-26T19:50:37.130514+0000","last_became_active":"2024-04-26T19:50:31.107801+0000","last_became_peered":"2024-04-26T19:50:31.107801+0000","last_unstale":"2024-04-26T19:50:37.130514+0000","last_undegraded":"2024-04-26T19:50:37.130514+0000","last_fullsized":"2024-04-26T19:50:37.130514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130138+0000","last_change":"2024-04-26T19:50:31.103841+0000","last_active":"2024-04-26T19:50:37.130138+0000","last_peered":"2024-04-26T19:50:37.130138+0000","last_clean":"2024-04-26T19:50:37.130138+0000","last_became_active":"2024-04-26T19:50:31.103366+0000","last_became_peered":"2024-04-26T19:50:31.103366+0000","last_unstale":"2024-04-26T19:50:37.130138+0000","last_undegraded":"2024-04-26T19:50:37.130138+0000","last_fullsized":"2024-04-26T19:50:37.130138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:22.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653217+0000","last_change":"2024-04-26T19:50:31.113740+0000","last_active":"2024-04-26T19:50:38.653217+0000","last_peered":"2024-04-26T19:50:38.653217+0000","last_clean":"2024-04-26T19:50:38.653217+0000","last_became_active":"2024-04-26T19:50:31.113431+0000","last_became_peered":"2024-04-26T19:50:31.113431+0000","last_unstale":"2024-04-26T19:50:38.653217+0000","last_undegraded":"2024-04-26T19:50:38.653217+0000","last_fullsized":"2024-04-26T19:50:38.653217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653594+0000","last_change":"2024-04-26T19:50:31.113198+0000","last_active":"2024-04-26T19:50:38.653594+0000","last_peered":"2024-04-26T19:50:38.653594+0000","last_clean":"2024-04-26T19:50:38.653594+0000","last_became_active":"2024-04-26T19:50:31.112582+0000","last_became_peered":"2024-04-26T19:50:31.112582+0000","last_unstale":"2024-04-26T19:50:38.653594+0000","last_undegraded":"2024-04-26T19:50:38.653594+0000","last_fullsized":"2024-04-26T19:50:38.653594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653995+0000","last_change":"2024-04-26T19:50:31.120475+0000","last_active":"2024-04-26T19:50:38.653995+0000","last_peered":"2024-04-26T19:50:38.653995+0000","last_clean":"2024-04-26T19:50:38.653995+0000","last_became_active":"2024-04-26T19:50:31.118576+0000","last_became_peered":"2024-04-26T19:50:31.118576+0000","last_unstale":"2024-04-26T19:50:38.653995+0000","last_undegraded":"2024-04-26T19:50:38.653995+0000","last_fullsized":"2024-04-26T19:50:38.653995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.291653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130241+0000","last_change":"2024-04-26T19:50:31.111907+0000","last_active":"2024-04-26T19:50:37.130241+0000","last_peered":"2024-04-26T19:50:37.130241+0000","last_clean":"2024-04-26T19:50:37.130241+0000","last_became_active":"2024-04-26T19:50:31.111623+0000","last_became_peered":"2024-04-26T19:50:31.111623+0000","last_unstale":"2024-04-26T19:50:37.130241+0000","last_undegraded":"2024-04-26T19:50:37.130241+0000","last_fullsized":"2024-04-26T19:50:37.130241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652974+0000","last_change":"2024-04-26T19:50:31.112347+0000","last_active":"2024-04-26T19:50:38.652974+0000","last_peered":"2024-04-26T19:50:38.652974+0000","last_clean":"2024-04-26T19:50:38.652974+0000","last_became_active":"2024-04-26T19:50:31.112074+0000","last_became_peered":"2024-04-26T19:50:31.112074+0000","last_unstale":"2024-04-26T19:50:38.652974+0000","last_undegraded":"2024-04-26T19:50:38.652974+0000","last_fullsized":"2024-04-26T19:50:38.652974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654050+0000","last_change":"2024-04-26T19:50:31.115255+0000","last_active":"2024-04-26T19:50:38.654050+0000","last_peered":"2024-04-26T19:50:38.654050+0000","last_clean":"2024-04-26T19:50:38.654050+0000","last_became_active":"2024-04-26T19:50:31.114991+0000","last_became_peered":"2024-04-26T19:50:31.114991+0000","last_unstale":"2024-04-26T19:50:38.654050+0000","last_undegraded":"2024-04-26T19:50:38.654050+0000","last_fullsized":"2024-04-26T19:50:38.654050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:39.941264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653558+0000","last_change":"2024-04-26T19:50:31.111763+0000","last_active":"2024-04-26T19:50:38.653558+0000","last_peered":"2024-04-26T19:50:38.653558+0000","last_clean":"2024-04-26T19:50:38.653558+0000","last_became_active":"2024-04-26T19:50:31.111487+0000","last_became_peered":"2024-04-26T19:50:31.111487+0000","last_unstale":"2024-04-26T19:50:38.653558+0000","last_undegraded":"2024-04-26T19:50:38.653558+0000","last_fullsized":"2024-04-26T19:50:38.653558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:13.577254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652454+0000","last_change":"2024-04-26T19:50:31.120637+0000","last_active":"2024-04-26T19:50:38.652454+0000","last_peered":"2024-04-26T19:50:38.652454+0000","last_clean":"2024-04-26T19:50:38.652454+0000","last_became_active":"2024-04-26T19:50:31.120388+0000","last_became_peered":"2024-04-26T19:50:31.120388+0000","last_unstale":"2024-04-26T19:50:38.652454+0000","last_undegraded":"2024-04-26T19:50:38.652454+0000","last_fullsized":"2024-04-26T19:50:38.652454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299955+0000","last_change":"2024-04-26T19:50:31.104484+0000","last_active":"2024-04-26T19:50:38.299955+0000","last_peered":"2024-04-26T19:50:38.299955+0000","last_clean":"2024-04-26T19:50:38.299955+0000","last_became_active":"2024-04-26T19:50:31.104172+0000","last_became_peered":"2024-04-26T19:50:31.104172+0000","last_unstale":"2024-04-26T19:50:38.299955+0000","last_undegraded":"2024-04-26T19:50:38.299955+0000","last_fullsized":"2024-04-26T19:50:38.299955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997884+0000","last_change":"2024-04-26T19:50:31.110923+0000","last_active":"2024-04-26T19:50:37.997884+0000","last_peered":"2024-04-26T19:50:37.997884+0000","last_clean":"2024-04-26T19:50:37.997884+0000","last_became_active":"2024-04-26T19:50:31.110601+0000","last_became_peered":"2024-04-26T19:50:31.110601+0000","last_unstale":"2024-04-26T19:50:37.997884+0000","last_undegraded":"2024-04-26T19:50:37.997884+0000","last_fullsized":"2024-04-26T19:50:37.997884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652978+0000","last_change":"2024-04-26T19:50:31.112465+0000","last_active":"2024-04-26T19:50:38.652978+0000","last_peered":"2024-04-26T19:50:38.652978+0000","last_clean":"2024-04-26T19:50:38.652978+0000","last_became_active":"2024-04-26T19:50:31.112186+0000","last_became_peered":"2024-04-26T19:50:31.112186+0000","last_unstale":"2024-04-26T19:50:38.652978+0000","last_undegraded":"2024-04-26T19:50:38.652978+0000","last_fullsized":"2024-04-26T19:50:38.652978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:10.399495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:50:54.485 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T19:50:54.485 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T19:50:54.485 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T19:50:54.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T19:50:54.486 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T19:50:54.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.616+0000 7f3531259640 1 Processor -- start 2024-04-26T19:50:54.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.616+0000 7f3531259640 1 -- start start 2024-04-26T19:50:54.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f3531259640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c148fd0 0x7f352c0d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f3531259640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c0b1290 con 0x7f352c06fb20 2024-04-26T19:50:54.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f3531259640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f352c0d4090 con 0x7f352c06ff00 2024-04-26T19:50:54.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f3531259640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f352c0d4370 con 0x7f352c148fd0 2024-04-26T19:50:54.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f352ad9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f352c06fb20 0x7f352c1488c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46254/0 (socket says 172.21.15.79:46254) 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f352ad9d640 1 -- 172.21.15.79:0/2278692584 learned_addr learned my addr 172.21.15.79:0/2278692584 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f352b59e640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c148fd0 0x7f352c0d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504303982 0 0) 0x7f352c0b1290 con 0x7f352c06fb20 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3514003520 con 0x7f352c06fb20 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.617+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2724690944 0 0) 0x7f352c0d4090 con 0x7f352c06ff00 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f352c0b1290 con 0x7f352c06ff00 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3458593948 0 0) 0x7f3514003520 con 0x7f352c06fb20 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c148fd0 msgr2=0x7f352c0d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c148fd0 0x7f352c0d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 >> v1:172.21.15.83:6789/0 conn(0x7f352c06ff00 legacy=0x7f352c0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c0d4650 con 0x7f352c06fb20 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f351c0019f0 con 0x7f352c06fb20 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 >> v1:172.21.15.79:6789/0 conn(0x7f352c06fb20 legacy=0x7f352c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f3514003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3514004800 0x7f3514004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35140051c0 0x7f351400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f352c0d4370 con 0x7f3514003b70 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f351400bf60 con 0x7f3514004800 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f351400c160 con 0x7f35140051c0 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f352b59e640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c148fd0 0x7f352c0d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f352ad9d640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f3514003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f352b59e640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35140051c0 0x7f351400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f352a59c640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3514004800 0x7f3514004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f352ad9d640 1 -- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35140051c0 msgr2=0x7f351400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f352ad9d640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35140051c0 0x7f351400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f352ad9d640 1 -- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3514004800 msgr2=0x7f3514004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f352ad9d640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3514004800 0x7f3514004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.618+0000 7f352ad9d640 1 -- 172.21.15.79:0/2278692584 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3514003520 con 0x7f3514003b70 2024-04-26T19:50:54.622 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f352ad9d640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f3514003f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f351c00a730 tx=0x7f3514005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5f7d67a4c9eca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c00a350 con 0x7f3514003b70 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351c02f070 con 0x7f3514003b70 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f3529d9b640 1 -- 172.21.15.79:0/2278692584 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c02bb90 con 0x7f3514003b70 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f3531259640 1 -- 172.21.15.79:0/2278692584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 msgr2=0x7f3514003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f3531259640 1 --2- 172.21.15.79:0/2278692584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f3514003f70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f351c00a730 tx=0x7f3514005830 comp rx=0 tx=0).stop 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f352b59e640 1 -- 172.21.15.79:0/2278692584 reap_dead start 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f3531259640 1 -- 172.21.15.79:0/2278692584 shutdown_connections 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f3531259640 1 -- 172.21.15.79:0/2278692584 >> 172.21.15.79:0/2278692584 conn(0x7f352c071c30 msgr2=0x7f352c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f3531259640 1 -- 172.21.15.79:0/2278692584 shutdown_connections 2024-04-26T19:50:54.623 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.619+0000 7f3531259640 1 -- 172.21.15.79:0/2278692584 wait complete. 2024-04-26T19:50:54.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.621+0000 7f3531259640 1 Processor -- start 2024-04-26T19:50:54.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.621+0000 7f3531259640 1 -- start start 2024-04-26T19:50:54.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f3531259640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f352c0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f3531259640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c13f250 0x7f352c13f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f3531259640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c142f00 0x7f352c143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f3531259640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f352c0d2cd0 con 0x7f3514003b70 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f3531259640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f352c0d4c10 con 0x7f352c13f250 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f3531259640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f352c0d2e50 con 0x7f352c142f00 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352ad9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f352c0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352ad9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f352c0d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38716/0 (socket says 172.21.15.79:38716) 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352ad9d640 1 -- 172.21.15.79:0/3303427517 learned_addr learned my addr 172.21.15.79:0/3303427517 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352a59c640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c13f250 0x7f352c13f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352b59e640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c142f00 0x7f352c143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352ad9d640 1 -- 172.21.15.79:0/3303427517 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c142f00 msgr2=0x7f352c143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352ad9d640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c142f00 0x7f352c143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352ad9d640 1 -- 172.21.15.79:0/3303427517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c13f250 msgr2=0x7f352c13f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352ad9d640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c13f250 0x7f352c13f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352ad9d640 1 -- 172.21.15.79:0/3303427517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f352c15fba0 con 0x7f3514003b70 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f352ad9d640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f352c0d1280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f351c00a730 tx=0x7f352c0d4820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f35137fe640 1 -- 172.21.15.79:0/3303427517 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c044070 con 0x7f3514003b70 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f35137fe640 1 -- 172.21.15.79:0/3303427517 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f351c02ca70 con 0x7f3514003b70 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f352c15fe80 con 0x7f3514003b70 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f35137fe640 1 -- 172.21.15.79:0/3303427517 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c00a350 con 0x7f3514003b70 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.622+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f352c160440 con 0x7f3514003b70 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.623+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f352c145f20 con 0x7f3514003b70 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.624+0000 7f35137fe640 1 -- 172.21.15.79:0/3303427517 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f351c047070 con 0x7f3514003b70 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.624+0000 7f35137fe640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f351806c8f0 0x7f351806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.624+0000 7f35137fe640 1 -- 172.21.15.79:0/3303427517 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f351c0abfa0 con 0x7f3514003b70 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.624+0000 7f352a59c640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f351806c8f0 0x7f351806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.625+0000 7f352a59c640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f351806c8f0 0x7f351806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3520000c00 tx=0x7f3520009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:54.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.628+0000 7f35137fe640 1 -- 172.21.15.79:0/3303427517 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f352c145f20 con 0x7f3514003b70 2024-04-26T19:50:54.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.848+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f352c0506c0 con 0x7f3514003b70 2024-04-26T19:50:54.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.851+0000 7f35137fe640 1 -- 172.21.15.79:0/3303427517 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174135 (secure 0 0 0) 0x7f351c02ce20 con 0x7f3514003b70 2024-04-26T19:50:54.854 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:54.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f351806c8f0 msgr2=0x7f351806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f351806c8f0 0x7f351806edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3520000c00 tx=0x7f3520009040 comp rx=0 tx=0).stop 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 msgr2=0x7f352c0d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f352c0d1280 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f351c00a730 tx=0x7f352c0d4820 comp rx=0 tx=0).stop 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 shutdown_connections 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f351806c8f0 0x7f351806edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f352c142f00 0x7f352c143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f352c13f250 0x7f352c13f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 --2- 172.21.15.79:0/3303427517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3514003b70 0x7f352c0d1280 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 >> 172.21.15.79:0/3303427517 conn(0x7f352c071c30 msgr2=0x7f352c146120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 shutdown_connections 2024-04-26T19:50:54.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.854+0000 7f3531259640 1 -- 172.21.15.79:0/3303427517 wait complete. 2024-04-26T19:50:54.865 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6832","nonce":3161939478},{"type":"v1","addr":"172.21.15.79:6833","nonce":3161939478}]},"active_addr":"172.21.15.79:6833/3161939478","active_change":"2024-04-26T19:50:10.244790+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":3742475428}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":16732380}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":1948040587}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.79:0","nonce":1084811462}]}]} 2024-04-26T19:50:54.870 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T19:50:54.870 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T19:50:54.870 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:54.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.966+0000 7f03a3646640 1 Processor -- start 2024-04-26T19:50:54.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.967+0000 7f03a3646640 1 -- start start 2024-04-26T19:50:54.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.967+0000 7f03a3646640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f039c13fee0 0x7f039c1432c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.967+0000 7f03a3646640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f039c0b4250 0x7f039c0d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.967+0000 7f03a3646640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f039c143800 con 0x7f039c1402c0 2024-04-26T19:50:54.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.967+0000 7f03a3646640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f039c0aa880 con 0x7f039c13fee0 2024-04-26T19:50:54.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.967+0000 7f03a3646640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f039c0aab60 con 0x7f039c0b4250 2024-04-26T19:50:54.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.968+0000 7f03a13e2640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f039c13fee0 0x7f039c1432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.968+0000 7f03a0be1640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f039c1402c0 0x7f039c0b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46262/0 (socket says 172.21.15.79:46262) 2024-04-26T19:50:54.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.968+0000 7f03a0be1640 1 -- 172.21.15.79:0/4276585182 learned_addr learned my addr 172.21.15.79:0/4276585182 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:54.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.968+0000 7f03a1be3640 1 --2- 172.21.15.79:0/4276585182 >> v2:172.21.15.83:3300/0 conn(0x7f039c0b4250 0x7f039c0d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.968+0000 7f03a13e2640 1 -- 172.21.15.79:0/4276585182 >> v2:172.21.15.83:3300/0 conn(0x7f039c0b4250 msgr2=0x7f039c0d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:54.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.968+0000 7f03a13e2640 1 --2- 172.21.15.79:0/4276585182 >> v2:172.21.15.83:3300/0 conn(0x7f039c0b4250 0x7f039c0d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.968+0000 7f03a13e2640 1 -- 172.21.15.79:0/4276585182 >> v1:172.21.15.79:6789/0 conn(0x7f039c1402c0 legacy=0x7f039c0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.968+0000 7f03a13e2640 1 -- 172.21.15.79:0/4276585182 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c0aae40 con 0x7f039c13fee0 2024-04-26T19:50:54.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.968+0000 7f0393fff640 1 -- 172.21.15.79:0/4276585182 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 892563247 0 0) 0x7f039c143800 con 0x7f039c1402c0 2024-04-26T19:50:54.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f03a13e2640 1 --2- 172.21.15.79:0/4276585182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 0x7f039c1432c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f039c15d810 tx=0x7f038c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c236862e0a88cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:54.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f0393fff640 1 -- 172.21.15.79:0/4276585182 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c037070 con 0x7f039c13fee0 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f0393fff640 1 -- 172.21.15.79:0/4276585182 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f038c032440 con 0x7f039c13fee0 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f0393fff640 1 -- 172.21.15.79:0/4276585182 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038c032810 con 0x7f039c13fee0 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f03a3646640 1 -- 172.21.15.79:0/4276585182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 msgr2=0x7f039c1432c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f03a3646640 1 --2- 172.21.15.79:0/4276585182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 0x7f039c1432c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f039c15d810 tx=0x7f038c02a1a0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f03a3646640 1 -- 172.21.15.79:0/4276585182 shutdown_connections 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f03a3646640 1 --2- 172.21.15.79:0/4276585182 >> v2:172.21.15.83:3300/0 conn(0x7f039c0b4250 0x7f039c0d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f03a3646640 1 --2- 172.21.15.79:0/4276585182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 0x7f039c1432c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.969+0000 7f03a3646640 1 -- 172.21.15.79:0/4276585182 >> 172.21.15.79:0/4276585182 conn(0x7f039c0b2720 msgr2=0x7f039c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.970+0000 7f03a3646640 1 -- 172.21.15.79:0/4276585182 shutdown_connections 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.970+0000 7f03a3646640 1 -- 172.21.15.79:0/4276585182 wait complete. 2024-04-26T19:50:54.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a3646640 1 Processor -- start 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a3646640 1 -- start start 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a3646640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f039c0b4250 0x7f039c13d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a3646640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 0x7f039c0b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a3646640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f039c1402c0 0x7f039c15f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a3646640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0d1500 con 0x7f039c13fee0 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a3646640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f039c0ab400 con 0x7f039c0b4250 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a3646640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f039c0ab920 con 0x7f039c1402c0 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a0be1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 0x7f039c0b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a0be1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 0x7f039c0b14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38724/0 (socket says 172.21.15.79:38724) 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.971+0000 7f03a0be1640 1 -- 172.21.15.79:0/3953281105 learned_addr learned my addr 172.21.15.79:0/3953281105 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a1be3640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f039c1402c0 0x7f039c15f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a13e2640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f039c0b4250 0x7f039c13d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a0be1640 1 -- 172.21.15.79:0/3953281105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f039c1402c0 msgr2=0x7f039c15f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a0be1640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f039c1402c0 0x7f039c15f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a0be1640 1 -- 172.21.15.79:0/3953281105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f039c0b4250 msgr2=0x7f039c13d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:54.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a0be1640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f039c0b4250 0x7f039c13d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:54.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a0be1640 1 -- 172.21.15.79:0/3953281105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c156d00 con 0x7f039c13fee0 2024-04-26T19:50:54.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a0be1640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 0x7f039c0b14e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f039800c410 tx=0x7f039800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:54.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f0391ffb640 1 -- 172.21.15.79:0/3953281105 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03980074e0 con 0x7f039c13fee0 2024-04-26T19:50:54.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f0391ffb640 1 -- 172.21.15.79:0/3953281105 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03980047e0 con 0x7f039c13fee0 2024-04-26T19:50:54.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c0af860 con 0x7f039c13fee0 2024-04-26T19:50:54.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f0391ffb640 1 -- 172.21.15.79:0/3953281105 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398004bd0 con 0x7f039c13fee0 2024-04-26T19:50:54.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.972+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c0afde0 con 0x7f039c13fee0 2024-04-26T19:50:54.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.973+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c0506c0 con 0x7f039c13fee0 2024-04-26T19:50:54.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.973+0000 7f0391ffb640 1 -- 172.21.15.79:0/3953281105 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0398011070 con 0x7f039c13fee0 2024-04-26T19:50:54.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.974+0000 7f0391ffb640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f038006c8f0 0x7f038006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:54.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.974+0000 7f03a13e2640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f038006c8f0 0x7f038006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:54.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.974+0000 7f0391ffb640 1 -- 172.21.15.79:0/3953281105 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f039808b8a0 con 0x7f039c13fee0 2024-04-26T19:50:54.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.974+0000 7f03a13e2640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f038006c8f0 0x7f038006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f038c034000 tx=0x7f038c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:54.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:54.978+0000 7f0391ffb640 1 -- 172.21.15.79:0/3953281105 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03980593d0 con 0x7f039c13fee0 2024-04-26T19:50:55.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.161+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f039c069bc0 con 0x7f039c13fee0 2024-04-26T19:50:55.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.162+0000 7f0391ffb640 1 -- 172.21.15.79:0/3953281105 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7f039805d6d0 con 0x7f039c13fee0 2024-04-26T19:50:55.163 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:55.163 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:55.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f038006c8f0 msgr2=0x7f038006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f038006c8f0 0x7f038006edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f038c034000 tx=0x7f038c02e000 comp rx=0 tx=0).stop 2024-04-26T19:50:55.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 msgr2=0x7f039c0b14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 0x7f039c0b14e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f039800c410 tx=0x7f039800c8d0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 shutdown_connections 2024-04-26T19:50:55.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f038006c8f0 0x7f038006edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f039c1402c0 0x7f039c15f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f039c13fee0 0x7f039c0b14e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 --2- 172.21.15.79:0/3953281105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f039c0b4250 0x7f039c13d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 >> 172.21.15.79:0/3953281105 conn(0x7f039c0b2720 msgr2=0x7f039c141300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 shutdown_connections 2024-04-26T19:50:55.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.164+0000 7f03a3646640 1 -- 172.21.15.79:0/3953281105 wait complete. 2024-04-26T19:50:55.177 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T19:50:55.178 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:55.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.273+0000 7fd520437640 1 Processor -- start 2024-04-26T19:50:55.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.273+0000 7fd520437640 1 -- start start 2024-04-26T19:50:55.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.274+0000 7fd520437640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806fb20 0x7fd5180d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.274+0000 7fd520437640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd51806ff00 0x7fd5180d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.274+0000 7fd520437640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5180b1290 con 0x7fd5181488c0 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.274+0000 7fd520437640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd5180d3e60 con 0x7fd51806ff00 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.274+0000 7fd520437640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd5180d4090 con 0x7fd51806fb20 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.274+0000 7fd51e1d3640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806fb20 0x7fd5180d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.275+0000 7fd51d9d2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd51806ff00 0x7fd5180d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.275+0000 7fd51e1d3640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806fb20 0x7fd5180d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44774/0 (socket says 172.21.15.79:44774) 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.275+0000 7fd51e1d3640 1 -- 172.21.15.79:0/3512845523 learned_addr learned my addr 172.21.15.79:0/3512845523 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.275+0000 7fd51d9d2640 1 --2- 172.21.15.79:0/3512845523 >> v2:172.21.15.83:3300/0 conn(0x7fd51806ff00 0x7fd5180d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:48316/0 (socket says 172.21.15.79:48316) 2024-04-26T19:50:55.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.275+0000 7fd51e1d3640 1 -- 172.21.15.79:0/3512845523 >> v2:172.21.15.83:3300/0 conn(0x7fd51806ff00 msgr2=0x7fd5180d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.275+0000 7fd51e1d3640 1 --2- 172.21.15.79:0/3512845523 >> v2:172.21.15.83:3300/0 conn(0x7fd51806ff00 0x7fd5180d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.275+0000 7fd51e1d3640 1 -- 172.21.15.79:0/3512845523 >> v1:172.21.15.83:6789/0 conn(0x7fd5181488c0 legacy=0x7fd518148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.275+0000 7fd51e1d3640 1 -- 172.21.15.79:0/3512845523 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5180d4370 con 0x7fd51806fb20 2024-04-26T19:50:55.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.275+0000 7fd51d9d2640 1 --2- 172.21.15.79:0/3512845523 >> v2:172.21.15.83:3300/0 conn(0x7fd51806ff00 0x7fd5180d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:55.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd51e1d3640 1 --2- 172.21.15.79:0/3512845523 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806fb20 0x7fd5180d2cd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd514009870 tx=0x7fd51402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8114e68402027e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd51d1d1640 1 -- 172.21.15.79:0/3512845523 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514037070 con 0x7fd51806fb20 2024-04-26T19:50:55.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd51d1d1640 1 -- 172.21.15.79:0/3512845523 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51402ae50 con 0x7fd51806fb20 2024-04-26T19:50:55.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd51d1d1640 1 -- 172.21.15.79:0/3512845523 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd514032860 con 0x7fd51806fb20 2024-04-26T19:50:55.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd520437640 1 -- 172.21.15.79:0/3512845523 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806fb20 msgr2=0x7fd5180d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd520437640 1 --2- 172.21.15.79:0/3512845523 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806fb20 0x7fd5180d2cd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd514009870 tx=0x7fd51402a300 comp rx=0 tx=0).stop 2024-04-26T19:50:55.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd520437640 1 -- 172.21.15.79:0/3512845523 shutdown_connections 2024-04-26T19:50:55.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd520437640 1 --2- 172.21.15.79:0/3512845523 >> v2:172.21.15.83:3300/0 conn(0x7fd51806ff00 0x7fd5180d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd520437640 1 --2- 172.21.15.79:0/3512845523 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806fb20 0x7fd5180d2cd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd520437640 1 -- 172.21.15.79:0/3512845523 >> 172.21.15.79:0/3512845523 conn(0x7fd518071c30 msgr2=0x7fd51806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd520437640 1 -- 172.21.15.79:0/3512845523 shutdown_connections 2024-04-26T19:50:55.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.276+0000 7fd520437640 1 -- 172.21.15.79:0/3512845523 wait complete. 2024-04-26T19:50:55.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd520437640 1 Processor -- start 2024-04-26T19:50:55.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd520437640 1 -- start start 2024-04-26T19:50:55.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd520437640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd51806fb20 0x7fd51813c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd520437640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806ff00 0x7fd51815e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd520437640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5181488c0 0x7fd518147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd520437640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd5180d4c10 con 0x7fd5181488c0 2024-04-26T19:50:55.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd520437640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd5180d4930 con 0x7fd51806fb20 2024-04-26T19:50:55.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd520437640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd5180aebb0 con 0x7fd51806ff00 2024-04-26T19:50:55.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd51e9d4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5181488c0 0x7fd518147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd51e9d4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5181488c0 0x7fd518147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38730/0 (socket says 172.21.15.79:38730) 2024-04-26T19:50:55.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd51e9d4640 1 -- 172.21.15.79:0/2444072298 learned_addr learned my addr 172.21.15.79:0/2444072298 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.278+0000 7fd51e1d3640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd51806fb20 0x7fd51813c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd51d9d2640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806ff00 0x7fd51815e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd51e9d4640 1 -- 172.21.15.79:0/2444072298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806ff00 msgr2=0x7fd51815e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd51e9d4640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806ff00 0x7fd51815e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd51e9d4640 1 -- 172.21.15.79:0/2444072298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd51806fb20 msgr2=0x7fd51813c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd51e9d4640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd51806fb20 0x7fd51813c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd51e9d4640 1 -- 172.21.15.79:0/2444072298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd51813fbb0 con 0x7fd5181488c0 2024-04-26T19:50:55.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd51e9d4640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5181488c0 0x7fd518147c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd51000c6e0 tx=0x7fd51000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd50effd640 1 -- 172.21.15.79:0/2444072298 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510003b70 con 0x7fd5181488c0 2024-04-26T19:50:55.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd50effd640 1 -- 172.21.15.79:0/2444072298 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd510003d30 con 0x7fd5181488c0 2024-04-26T19:50:55.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd51813fe90 con 0x7fd5181488c0 2024-04-26T19:50:55.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd50effd640 1 -- 172.21.15.79:0/2444072298 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd510005310 con 0x7fd5181488c0 2024-04-26T19:50:55.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.279+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd51813bf30 con 0x7fd5181488c0 2024-04-26T19:50:55.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.280+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5180506c0 con 0x7fd5181488c0 2024-04-26T19:50:55.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.281+0000 7fd50effd640 1 -- 172.21.15.79:0/2444072298 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5100054b0 con 0x7fd5181488c0 2024-04-26T19:50:55.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.281+0000 7fd50effd640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd4fc06c8f0 0x7fd4fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.281+0000 7fd50effd640 1 -- 172.21.15.79:0/2444072298 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd51008bbe0 con 0x7fd5181488c0 2024-04-26T19:50:55.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.281+0000 7fd51e1d3640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd4fc06c8f0 0x7fd4fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.281+0000 7fd51e1d3640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd4fc06c8f0 0x7fd4fc06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd514034000 tx=0x7fd51402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.285+0000 7fd50effd640 1 -- 172.21.15.79:0/2444072298 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd510059680 con 0x7fd5181488c0 2024-04-26T19:50:55.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.460+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd51813c330 con 0x7fd5181488c0 2024-04-26T19:50:55.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.462+0000 7fd50effd640 1 -- 172.21.15.79:0/2444072298 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7fd51005d980 con 0x7fd5181488c0 2024-04-26T19:50:55.462 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:55.463 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.463+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd4fc06c8f0 msgr2=0x7fd4fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.463+0000 7fd520437640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd4fc06c8f0 0x7fd4fc06edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd514034000 tx=0x7fd51402e000 comp rx=0 tx=0).stop 2024-04-26T19:50:55.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.463+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5181488c0 msgr2=0x7fd518147c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.463+0000 7fd520437640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5181488c0 0x7fd518147c70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd51000c6e0 tx=0x7fd51000cba0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.463+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 shutdown_connections 2024-04-26T19:50:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.463+0000 7fd520437640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd4fc06c8f0 0x7fd4fc06edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.463+0000 7fd520437640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd5181488c0 0x7fd518147c70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.463+0000 7fd520437640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd51806ff00 0x7fd51815e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.463+0000 7fd520437640 1 --2- 172.21.15.79:0/2444072298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd51806fb20 0x7fd51813c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.464+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 >> 172.21.15.79:0/2444072298 conn(0x7fd518071c30 msgr2=0x7fd518160ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.464+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 shutdown_connections 2024-04-26T19:50:55.465 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.464+0000 7fd520437640 1 -- 172.21.15.79:0/2444072298 wait complete. 2024-04-26T19:50:55.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T19:50:55.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T19:50:55.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T19:50:55.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-26T19:50:55.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-26T19:50:55.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-26T19:50:55.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-26T19:50:55.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-26T19:50:55.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.641+0000 7fcf3c46d640 1 Processor -- start 2024-04-26T19:50:55.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.641+0000 7fcf3c46d640 1 -- start start 2024-04-26T19:50:55.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fcf3c46d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcf340738a0 0x7fcf3416efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fbde759e640 1 Processor -- start 2024-04-26T19:50:55.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fbde759e640 1 -- start start 2024-04-26T19:50:55.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fcf3c46d640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf340cd490 con 0x7fcf34071f20 2024-04-26T19:50:55.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fcf3c46d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf340cd770 con 0x7fcf3416f500 2024-04-26T19:50:55.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fcf3c46d640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fcf34072690 con 0x7fcf340738a0 2024-04-26T19:50:55.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fbb16207640 1 Processor -- start 2024-04-26T19:50:55.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fcf3aa0a640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fcf34071f20 0x7fcf340cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46276/0 (socket says 172.21.15.79:46276) 2024-04-26T19:50:55.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fcf3aa0a640 1 -- 172.21.15.79:0/754973500 learned_addr learned my addr 172.21.15.79:0/754973500 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:55.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fcf3a209640 1 --2- 172.21.15.79:0/754973500 >> v2:172.21.15.79:3300/0 conn(0x7fcf340738a0 0x7fcf3416efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.644+0000 7feb763da640 1 Processor -- start 2024-04-26T19:50:55.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.644+0000 7feb763da640 1 -- start start 2024-04-26T19:50:55.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.644+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2782640762 0 0) 0x7fcf340cd490 con 0x7fcf34071f20 2024-04-26T19:50:55.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.644+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf20003440 con 0x7fcf34071f20 2024-04-26T19:50:55.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.644+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1702024950 0 0) 0x7fcf20003440 con 0x7fcf34071f20 2024-04-26T19:50:55.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.644+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 >> v2:172.21.15.79:3300/0 conn(0x7fcf340738a0 msgr2=0x7fcf3416efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.644+0000 7fcf39207640 1 --2- 172.21.15.79:0/754973500 >> v2:172.21.15.79:3300/0 conn(0x7fcf340738a0 0x7fcf3416efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.644+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 >> v1:172.21.15.83:6789/0 conn(0x7fcf3416f500 legacy=0x7fcf34071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.643+0000 7fbb16207640 1 -- start start 2024-04-26T19:50:55.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbde759e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde8155910 0x7fbde80d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb16207640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbb080ead00 0x7fbb0804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb16207640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb08006ef0 0x7fbb08007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbde759e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb16207640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb08053720 con 0x7fbb0804ffd0 2024-04-26T19:50:55.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf340cef00 con 0x7fcf34071f20 2024-04-26T19:50:55.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fcf2c005050 con 0x7fcf34071f20 2024-04-26T19:50:55.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 >> v1:172.21.15.79:6789/0 conn(0x7fcf34071f20 legacy=0x7fcf340cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fcf39207640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf20003aa0 0x7fcf20003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fcf39207640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf200088a0 0x7fcf2000ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fcf39207640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 0x7fcf2000da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2000e320 con 0x7fcf20003aa0 2024-04-26T19:50:55.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2000e4d0 con 0x7fcf2000b5b0 2024-04-26T19:50:55.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fcf2000e6d0 con 0x7fcf200088a0 2024-04-26T19:50:55.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fdd82f64640 1 Processor -- start 2024-04-26T19:50:55.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb16207640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fbb080078a0 con 0x7fbb080ead00 2024-04-26T19:50:55.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb16207640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbb08054330 con 0x7fbb08006ef0 2024-04-26T19:50:55.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbde759e640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbde8155cf0 con 0x7fbde815b830 2024-04-26T19:50:55.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbde759e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbde80b27b0 con 0x7fbde8155910 2024-04-26T19:50:55.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbde759e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbde8071810 con 0x7fbde80b1e00 2024-04-26T19:50:55.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbde659c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbde8155910 0x7fbde80d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb0f7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbb080ead00 0x7fbb0804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb0f7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbb080ead00 0x7fbb0804fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38736/0 (socket says 172.21.15.79:38736) 2024-04-26T19:50:55.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb0f7fe640 1 -- 172.21.15.79:0/2394842818 learned_addr learned my addr 172.21.15.79:0/2394842818 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbde6d9d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbde6d9d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44784/0 (socket says 172.21.15.79:44784) 2024-04-26T19:50:55.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbde6d9d640 1 -- 172.21.15.79:0/3517117957 learned_addr learned my addr 172.21.15.79:0/3517117957 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb0ffff640 1 --2- 172.21.15.79:0/2394842818 >> v2:172.21.15.83:3300/0 conn(0x7fbb08006ef0 0x7fbb08007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb0ffff640 1 -- 172.21.15.79:0/2394842818 >> v2:172.21.15.79:3300/0 conn(0x7fbb080ead00 msgr2=0x7fbb0804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb0ffff640 1 --2- 172.21.15.79:0/2394842818 >> v2:172.21.15.79:3300/0 conn(0x7fbb080ead00 0x7fbb0804fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb0ffff640 1 -- 172.21.15.79:0/2394842818 >> v1:172.21.15.83:6789/0 conn(0x7fbb0804ffd0 legacy=0x7fbb08037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fbb0ffff640 1 -- 172.21.15.79:0/2394842818 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb08053a00 con 0x7fbb08006ef0 2024-04-26T19:50:55.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7f83fa4cf640 1 Processor -- start 2024-04-26T19:50:55.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fcf3aa0a640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 0x7fcf2000da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fcf3a209640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf20003aa0 0x7fcf20003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fbde659c640 1 -- 172.21.15.79:0/3517117957 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 msgr2=0x7fbde80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fbde659c640 1 --2- 172.21.15.79:0/3517117957 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fcf39a08640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf200088a0 0x7fcf2000ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fbde659c640 1 -- 172.21.15.79:0/3517117957 >> v1:172.21.15.79:6789/0 conn(0x7fbde815b830 legacy=0x7fbde80b0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fbde659c640 1 -- 172.21.15.79:0/3517117957 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde815bdd0 con 0x7fbde8155910 2024-04-26T19:50:55.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.645+0000 7fdd82f64640 1 -- start start 2024-04-26T19:50:55.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fcf3aa0a640 1 -- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf200088a0 msgr2=0x7fcf2000ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fcf3aa0a640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf200088a0 0x7fcf2000ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7fcf3aa0a640 1 -- 172.21.15.79:0/754973500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf20003aa0 msgr2=0x7fcf20003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7f83fa4cf640 1 -- start start 2024-04-26T19:50:55.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f99fffff640 1 Processor -- start 2024-04-26T19:50:55.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f45c610a640 1 Processor -- start 2024-04-26T19:50:55.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7feb763da640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce230 0x7feb7015bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7feb763da640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7feb700c4d20 0x7feb700c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce230 0x7feb7015bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce230 0x7feb7015bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44814/0 (socket says 172.21.15.79:44814) 2024-04-26T19:50:55.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7feb6ffff640 1 -- 172.21.15.79:0/413740105 learned_addr learned my addr 172.21.15.79:0/413740105 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7feb763da640 1 -- 172.21.15.79:0/413740105 --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb7015c310 con 0x7feb700ce610 2024-04-26T19:50:55.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7feb763da640 1 -- 172.21.15.79:0/413740105 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7feb700c57e0 con 0x7feb700c4d20 2024-04-26T19:50:55.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.646+0000 7feb763da640 1 -- 172.21.15.79:0/413740105 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7feb700c5ac0 con 0x7feb700ce230 2024-04-26T19:50:55.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7feb6ffff640 1 -- 172.21.15.79:0/413740105 >> v2:172.21.15.79:3300/0 conn(0x7feb700c4d20 msgr2=0x7feb700c5180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:50:55.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd82f64640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd7c0c6e30 0x7fdd7c152550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd81f62640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd7c0c6e30 0x7fdd7c152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f99fffff640 1 -- start start 2024-04-26T19:50:55.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd81f62640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd7c0c6e30 0x7fdd7c152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38766/0 (socket says 172.21.15.79:38766) 2024-04-26T19:50:55.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd81761640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdd7c15c790 0x7fdd7c0d0c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46298/0 (socket says 172.21.15.79:46298) 2024-04-26T19:50:55.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fbb0ffff640 1 --2- 172.21.15.79:0/2394842818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb08006ef0 0x7fbb08007360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb10051930 tx=0x7fbb100a6120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1debe3c55bbe47ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd82f64640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd7c071850 0x7fdd7c0c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f83fa4cf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83f40d0940 0x7f83f40b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fbde659c640 1 --2- 172.21.15.79:0/3517117957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde8155910 0x7fbde80d18a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d90 tx=0x7fbde002a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=704ebf6b42b91305 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7feb6f7fe640 1 --2- 172.21.15.79:0/413740105 >> v2:172.21.15.79:3300/0 conn(0x7feb700c4d20 0x7feb700c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fbb0e7fc640 1 -- 172.21.15.79:0/2394842818 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100ae410 con 0x7fbb08006ef0 2024-04-26T19:50:55.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fbb0e7fc640 1 -- 172.21.15.79:0/2394842818 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb100aa070 con 0x7fbb08006ef0 2024-04-26T19:50:55.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fbb0e7fc640 1 -- 172.21.15.79:0/2394842818 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100ae830 con 0x7fbb08006ef0 2024-04-26T19:50:55.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd81761640 1 -- 172.21.15.79:0/3585552452 learned_addr learned my addr 172.21.15.79:0/3585552452 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:55.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd82763640 1 --2- 172.21.15.79:0/3585552452 >> v2:172.21.15.83:3300/0 conn(0x7fdd7c071850 0x7fdd7c0c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f83f3fff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f83f415dde0 0x7f83f4167460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:51086/0 (socket says 172.21.15.79:51086) 2024-04-26T19:50:55.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f83f3fff640 1 -- 172.21.15.79:0/2525076092 learned_addr learned my addr 172.21.15.79:0/2525076092 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:55.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd82f64640 1 -- 172.21.15.79:0/3585552452 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd7c0c6070 con 0x7fdd7c15c790 2024-04-26T19:50:55.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f45c610a640 1 -- start start 2024-04-26T19:50:55.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd82f64640 1 -- 172.21.15.79:0/3585552452 --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7c0d1470 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fdd82f64640 1 -- 172.21.15.79:0/3585552452 --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdd7c0c6350 con 0x7fdd7c071850 2024-04-26T19:50:55.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fcf3aa0a640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf20003aa0 0x7fcf20003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fbde559a640 1 -- 172.21.15.79:0/3517117957 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0031600 con 0x7fbde8155910 2024-04-26T19:50:55.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fbde559a640 1 -- 172.21.15.79:0/3517117957 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde00317a0 con 0x7fbde8155910 2024-04-26T19:50:55.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fbde559a640 1 -- 172.21.15.79:0/3517117957 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0031b70 con 0x7fbde8155910 2024-04-26T19:50:55.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fcf3aa0a640 1 -- 172.21.15.79:0/754973500 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf340cef00 con 0x7fcf2000b5b0 2024-04-26T19:50:55.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fcf3a209640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf20003aa0 0x7fcf20003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:55.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7fcf39a08640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf200088a0 0x7fcf2000ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:55.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fcf3aa0a640 1 -- 172.21.15.79:0/754973500 reap_dead start 2024-04-26T19:50:55.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f45c610a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f45c015dde0 0x7f45c0167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fdd82763640 1 -- 172.21.15.79:0/3585552452 >> v2:172.21.15.79:3300/0 conn(0x7fdd7c0c6e30 msgr2=0x7fdd7c152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fdd82763640 1 --2- 172.21.15.79:0/3585552452 >> v2:172.21.15.79:3300/0 conn(0x7fdd7c0c6e30 0x7fdd7c152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f83fa4cf640 1 -- 172.21.15.79:0/2525076092 --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f40b0830 con 0x7f83f40d0dc0 2024-04-26T19:50:55.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fdd82763640 1 -- 172.21.15.79:0/3585552452 >> v1:172.21.15.79:6789/0 conn(0x7fdd7c15c790 legacy=0x7fdd7c0d0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fdd82763640 1 -- 172.21.15.79:0/3585552452 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c0c6630 con 0x7fdd7c071850 2024-04-26T19:50:55.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f99fffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f99f80e7600 0x7f99f8007670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f99fffff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f8053c50 0x7f99f803bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f99fffff640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f80e79e0 con 0x7f99f80589d0 2024-04-26T19:50:55.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f45c610a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f45c00d0940 0x7f45c00b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f45c610a640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c00b0830 con 0x7f45c00d0df0 2024-04-26T19:50:55.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f45c610a640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f45c00b0b10 con 0x7f45c015dde0 2024-04-26T19:50:55.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f45c610a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f45c00b0df0 con 0x7f45c00d0940 2024-04-26T19:50:55.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f99fffff640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f99f8007bb0 con 0x7f99f80e7600 2024-04-26T19:50:55.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f99fffff640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f99f80065d0 con 0x7f99f8053c50 2024-04-26T19:50:55.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f83fa4cf640 1 -- 172.21.15.79:0/2525076092 --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f40b0b10 con 0x7f83f415dde0 2024-04-26T19:50:55.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fcf3aa0a640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 0x7fcf2000da70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c001740 tx=0x7fcf20003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9202c5e5a03a7d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f83fa4cf640 1 -- 172.21.15.79:0/2525076092 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f83f4167b70 con 0x7f83f40d0940 2024-04-26T19:50:55.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7fdd81f62640 1 --2- 172.21.15.79:0/3585552452 >> v2:172.21.15.79:3300/0 conn(0x7fdd7c0c6e30 0x7fdd7c152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:55.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.647+0000 7f83f37fe640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83f40d0940 0x7f83f40b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f99ff7fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f8053c50 0x7f99f803bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f99ff7fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f8053c50 0x7f99f803bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44838/0 (socket says 172.21.15.79:44838) 2024-04-26T19:50:55.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f99ff7fe640 1 -- 172.21.15.79:0/1127690119 learned_addr learned my addr 172.21.15.79:0/1127690119 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f99feffd640 1 --2- 172.21.15.79:0/1127690119 >> v2:172.21.15.83:3300/0 conn(0x7f99f80e7600 0x7f99f8007670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7feb6ffff640 1 --2- 172.21.15.79:0/413740105 >> v2:172.21.15.79:3300/0 conn(0x7feb700c4d20 0x7feb700c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7feb6ffff640 1 -- 172.21.15.79:0/413740105 >> v1:172.21.15.83:6789/0 conn(0x7feb700ce610 legacy=0x7feb700ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7feb6ffff640 1 -- 172.21.15.79:0/413740105 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb700c5da0 con 0x7feb700ce230 2024-04-26T19:50:55.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7f45beffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f45c00d0940 0x7f45c00b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbb16207640 1 -- 172.21.15.79:0/2394842818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb08006ef0 msgr2=0x7fbb08007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbb16207640 1 --2- 172.21.15.79:0/2394842818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb08006ef0 0x7fbb08007360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fbb10051930 tx=0x7fbb100a6120 comp rx=0 tx=0).stop 2024-04-26T19:50:55.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45bffff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f45c00d0df0 0x7f45c01643c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46306/0 (socket says 172.21.15.79:46306) 2024-04-26T19:50:55.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbde759e640 1 -- 172.21.15.79:0/3517117957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde8155910 msgr2=0x7fbde80d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45bffff640 1 -- 172.21.15.79:0/4117061758 learned_addr learned my addr 172.21.15.79:0/4117061758 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:55.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbde759e640 1 --2- 172.21.15.79:0/3517117957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde8155910 0x7fbde80d18a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d90 tx=0x7fbde002a610 comp rx=0 tx=0).stop 2024-04-26T19:50:55.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f99ff7fe640 1 -- 172.21.15.79:0/1127690119 >> v2:172.21.15.83:3300/0 conn(0x7f99f80e7600 msgr2=0x7f99f8007670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f99ff7fe640 1 --2- 172.21.15.79:0/1127690119 >> v2:172.21.15.83:3300/0 conn(0x7f99f80e7600 0x7f99f8007670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f99ff7fe640 1 -- 172.21.15.79:0/1127690119 >> v1:172.21.15.83:6789/0 conn(0x7f99f80589d0 legacy=0x7f99f8053540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f99ff7fe640 1 -- 172.21.15.79:0/1127690119 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f80068b0 con 0x7f99f8053c50 2024-04-26T19:50:55.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45bf7fe640 1 --2- 172.21.15.79:0/4117061758 >> v2:172.21.15.79:3300/0 conn(0x7f45c015dde0 0x7f45c0167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbb16207640 1 -- 172.21.15.79:0/2394842818 shutdown_connections 2024-04-26T19:50:55.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbb16207640 1 --2- 172.21.15.79:0/2394842818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb08006ef0 0x7fbb08007360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbb16207640 1 --2- 172.21.15.79:0/2394842818 >> v2:172.21.15.79:3300/0 conn(0x7fbb080ead00 0x7fbb0804fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbb16207640 1 -- 172.21.15.79:0/2394842818 >> 172.21.15.79:0/2394842818 conn(0x7fbb08009fb0 msgr2=0x7fbb0800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2938986220 0 0) 0x7f83f40b0830 con 0x7f83f40d0dc0 2024-04-26T19:50:55.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83dc003520 con 0x7f83f40d0dc0 2024-04-26T19:50:55.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2051199995 0 0) 0x7f83f40b0b10 con 0x7f83f415dde0 2024-04-26T19:50:55.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83f40b0830 con 0x7f83f415dde0 2024-04-26T19:50:55.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbb16207640 1 -- 172.21.15.79:0/2394842818 shutdown_connections 2024-04-26T19:50:55.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.648+0000 7feb6f7fe640 1 --2- 172.21.15.79:0/413740105 >> v2:172.21.15.79:3300/0 conn(0x7feb700c4d20 0x7feb700c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:55.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7feb6ffff640 1 --2- 172.21.15.79:0/413740105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce230 0x7feb7015bdd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feb60009870 tx=0x7feb6002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35179cfeee2a9878 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7feb6effd640 1 -- 172.21.15.79:0/413740105 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60031450 con 0x7feb700ce230 2024-04-26T19:50:55.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7feb6effd640 1 -- 172.21.15.79:0/413740105 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb60036070 con 0x7feb700ce230 2024-04-26T19:50:55.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7feb6effd640 1 -- 172.21.15.79:0/413740105 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60031850 con 0x7feb700ce230 2024-04-26T19:50:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 <== mon.1 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 936069422 0 0) 0x7f83f40b0830 con 0x7f83f415dde0 2024-04-26T19:50:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83f40d0940 msgr2=0x7f83f40b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83f40d0940 0x7f83f40b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 >> v1:172.21.15.79:6789/0 conn(0x7f83f40d0dc0 legacy=0x7f83f41643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f40b0df0 con 0x7f83f415dde0 2024-04-26T19:50:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000b600 con 0x7fcf2000b5b0 2024-04-26T19:50:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c02f070 con 0x7fcf2000b5b0 2024-04-26T19:50:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fcf39207640 1 -- 172.21.15.79:0/754973500 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c008d60 con 0x7fcf2000b5b0 2024-04-26T19:50:55.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 <== mon.1 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f83e40015f0 con 0x7f83f415dde0 2024-04-26T19:50:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 >> v1:172.21.15.83:6789/0 conn(0x7f83f415dde0 legacy=0x7f83f4167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f83f2ffd640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83dc004ba0 0x7f83dc004fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fdd82763640 1 --2- 172.21.15.79:0/3585552452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd7c071850 0x7fdd7c0c4d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c070af0 tx=0x7fdd7402a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce9d74bbcc4356c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f2ffd640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83dc0037a0 0x7f83dc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f2ffd640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83dc004160 0x7f83dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4167b70 con 0x7f83dc004ba0 2024-04-26T19:50:55.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83dc00bfe0 con 0x7f83dc0037a0 2024-04-26T19:50:55.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f83dc00c1e0 con 0x7f83dc004160 2024-04-26T19:50:55.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7fbb16207640 1 -- 172.21.15.79:0/2394842818 wait complete. 2024-04-26T19:50:55.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7fdd80f60640 1 -- 172.21.15.79:0/3585552452 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74007740 con 0x7fdd7c071850 2024-04-26T19:50:55.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7fdd80f60640 1 -- 172.21.15.79:0/3585552452 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd740078e0 con 0x7fdd7c071850 2024-04-26T19:50:55.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7fdd80f60640 1 -- 172.21.15.79:0/3585552452 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74007cb0 con 0x7fdd7c071850 2024-04-26T19:50:55.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139879889 0 0) 0x7f45c00b0830 con 0x7f45c00d0df0 2024-04-26T19:50:55.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45a8003520 con 0x7f45c00d0df0 2024-04-26T19:50:55.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1869553116 0 0) 0x7f45a8003520 con 0x7f45c00d0df0 2024-04-26T19:50:55.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 >> v2:172.21.15.83:3300/0 conn(0x7f45c00d0940 msgr2=0x7f45c00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45be7fc640 1 --2- 172.21.15.79:0/4117061758 >> v2:172.21.15.83:3300/0 conn(0x7f45c00d0940 0x7f45c00b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 >> v2:172.21.15.79:3300/0 conn(0x7f45c015dde0 msgr2=0x7f45c0167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45be7fc640 1 --2- 172.21.15.79:0/4117061758 >> v2:172.21.15.79:3300/0 conn(0x7f45c015dde0 0x7f45c0167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7fbde759e640 1 -- 172.21.15.79:0/3517117957 shutdown_connections 2024-04-26T19:50:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c00b10d0 con 0x7f45c00d0df0 2024-04-26T19:50:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.649+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f45b4001670 con 0x7f45c00d0df0 2024-04-26T19:50:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 >> v1:172.21.15.79:6789/0 conn(0x7f45c00d0df0 legacy=0x7f45c01643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f45be7fc640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45a8003a00 0x7f45a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f45be7fc640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45a8004800 0x7f45a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f45be7fc640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45a80051c0 0x7f45a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7fbde759e640 1 --2- 172.21.15.79:0/3517117957 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7fbde759e640 1 --2- 172.21.15.79:0/3517117957 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde8155910 0x7fbde80d18a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7fbde759e640 1 -- 172.21.15.79:0/3517117957 >> 172.21.15.79:0/3517117957 conn(0x7fbde80b4e80 msgr2=0x7fbde806feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b0b10 con 0x7f45a80051c0 2024-04-26T19:50:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b0df0 con 0x7f45a8003a00 2024-04-26T19:50:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f45a800c110 con 0x7f45a8004800 2024-04-26T19:50:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f37fe640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83dc0037a0 0x7f83dc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f3fff640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83dc004ba0 0x7f83dc004fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f99ff7fe640 1 --2- 172.21.15.79:0/1127690119 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f8053c50 0x7f99f803bd40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9a0006b410 tx=0x7f9a000a61a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49119ffc23f78464 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f45beffd640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45a8004800 0x7f45a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7feb763da640 1 -- 172.21.15.79:0/413740105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce230 msgr2=0x7feb7015bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7feb763da640 1 --2- 172.21.15.79:0/413740105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce230 0x7feb7015bdd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feb60009870 tx=0x7feb6002a1a0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7fbde759e640 1 -- 172.21.15.79:0/3517117957 shutdown_connections 2024-04-26T19:50:55.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7fbde759e640 1 -- 172.21.15.79:0/3517117957 wait complete. 2024-04-26T19:50:55.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f99fdffb640 1 -- 172.21.15.79:0/1127690119 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a000ae410 con 0x7f99f8053c50 2024-04-26T19:50:55.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f99fdffb640 1 -- 172.21.15.79:0/1127690119 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a000aa070 con 0x7f99f8053c50 2024-04-26T19:50:55.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f99fdffb640 1 -- 172.21.15.79:0/1127690119 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a000ae830 con 0x7f99f8053c50 2024-04-26T19:50:55.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7feb763da640 1 -- 172.21.15.79:0/413740105 shutdown_connections 2024-04-26T19:50:55.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7feb763da640 1 --2- 172.21.15.79:0/413740105 >> v2:172.21.15.79:3300/0 conn(0x7feb700c4d20 0x7feb700c5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7feb763da640 1 --2- 172.21.15.79:0/413740105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce230 0x7feb7015bdd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7feb763da640 1 -- 172.21.15.79:0/413740105 >> 172.21.15.79:0/413740105 conn(0x7feb7006f580 msgr2=0x7feb700740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7feb763da640 1 -- 172.21.15.79:0/413740105 shutdown_connections 2024-04-26T19:50:55.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fbb16207640 1 Processor -- start 2024-04-26T19:50:55.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7feb763da640 1 -- 172.21.15.79:0/413740105 wait complete. 2024-04-26T19:50:55.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fdd82f64640 1 -- 172.21.15.79:0/3585552452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd7c071850 msgr2=0x7fdd7c0c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fdd82f64640 1 --2- 172.21.15.79:0/3585552452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd7c071850 0x7fdd7c0c4d20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c070af0 tx=0x7fdd7402a5f0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7f99fffff640 1 -- 172.21.15.79:0/1127690119 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f8053c50 msgr2=0x7f99f803bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7f99fffff640 1 --2- 172.21.15.79:0/1127690119 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f8053c50 0x7f99f803bd40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9a0006b410 tx=0x7f9a000a61a0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fcf3c46d640 1 -- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 msgr2=0x7fcf2000da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fcf3c46d640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 0x7fcf2000da70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c001740 tx=0x7fcf20003730 comp rx=0 tx=0).stop 2024-04-26T19:50:55.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fdd82f64640 1 -- 172.21.15.79:0/3585552452 shutdown_connections 2024-04-26T19:50:55.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fdd82f64640 1 --2- 172.21.15.79:0/3585552452 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd7c071850 0x7fdd7c0c4d20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fdd82f64640 1 --2- 172.21.15.79:0/3585552452 >> v2:172.21.15.79:3300/0 conn(0x7fdd7c0c6e30 0x7fdd7c152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fdd82f64640 1 -- 172.21.15.79:0/3585552452 >> 172.21.15.79:0/3585552452 conn(0x7fdd7c06eb00 msgr2=0x7fdd7c0740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fbb16207640 1 -- start start 2024-04-26T19:50:55.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7f45bffff640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45a80051c0 0x7f45a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7f45bf7fe640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45a8003a00 0x7f45a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fcf3c46d640 1 -- 172.21.15.79:0/754973500 shutdown_connections 2024-04-26T19:50:55.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fcf3c46d640 1 --2- 172.21.15.79:0/754973500 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 0x7fcf2000da70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fcf3c46d640 1 -- 172.21.15.79:0/754973500 >> 172.21.15.79:0/754973500 conn(0x7fcf340b4e80 msgr2=0x7fcf3406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f37fe640 1 -- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83dc004160 msgr2=0x7f83dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f37fe640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83dc004160 0x7f83dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f37fe640 1 -- 172.21.15.79:0/2525076092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83dc004ba0 msgr2=0x7f83dc004fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f37fe640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83dc004ba0 0x7f83dc004fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.650+0000 7f83f37fe640 1 -- 172.21.15.79:0/2525076092 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f40b0830 con 0x7f83dc0037a0 2024-04-26T19:50:55.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7f83f3fff640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83dc004ba0 0x7f83dc004fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:55.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fcf3c46d640 1 -- 172.21.15.79:0/754973500 shutdown_connections 2024-04-26T19:50:55.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fcf3c46d640 1 -- 172.21.15.79:0/754973500 wait complete. 2024-04-26T19:50:55.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fdd82f64640 1 -- 172.21.15.79:0/3585552452 shutdown_connections 2024-04-26T19:50:55.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7f99fffff640 1 -- 172.21.15.79:0/1127690119 shutdown_connections 2024-04-26T19:50:55.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7f99fffff640 1 --2- 172.21.15.79:0/1127690119 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f8053c50 0x7f99f803bd40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7f99fffff640 1 --2- 172.21.15.79:0/1127690119 >> v2:172.21.15.83:3300/0 conn(0x7f99f80e7600 0x7f99f8007670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7f99fffff640 1 -- 172.21.15.79:0/1127690119 >> 172.21.15.79:0/1127690119 conn(0x7f99f8058ef0 msgr2=0x7f99f8059300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f83f37fe640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83dc0037a0 0x7f83dc003c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009980 tx=0x7f83dc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ee8d25412f07879 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f99fffff640 1 -- 172.21.15.79:0/1127690119 shutdown_connections 2024-04-26T19:50:55.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f45beffd640 1 -- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45a8003a00 msgr2=0x7f45a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f45beffd640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45a8003a00 0x7f45a8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f45beffd640 1 -- 172.21.15.79:0/4117061758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45a80051c0 msgr2=0x7f45a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:55.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f45beffd640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45a80051c0 0x7f45a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.651+0000 7fdd82f64640 1 -- 172.21.15.79:0/3585552452 wait complete. 2024-04-26T19:50:55.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f45beffd640 1 -- 172.21.15.79:0/4117061758 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a8003520 con 0x7f45a8004800 2024-04-26T19:50:55.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f45bffff640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45a80051c0 0x7f45a800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:55.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f45bffff640 1 -- 172.21.15.79:0/4117061758 reap_dead start 2024-04-26T19:50:55.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f45bf7fe640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45a8003a00 0x7f45a8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:55.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7fbde759e640 1 Processor -- start 2024-04-26T19:50:55.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7fbde759e640 1 -- start start 2024-04-26T19:50:55.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7feb763da640 1 Processor -- start 2024-04-26T19:50:55.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7feb763da640 1 -- start start 2024-04-26T19:50:55.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fbb16207640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08006ef0 0x7fbb080e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fbb0f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08006ef0 0x7fbb080e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fbb0f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08006ef0 0x7fbb080e6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38806/0 (socket says 172.21.15.79:38806) 2024-04-26T19:50:55.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8036070 con 0x7f83dc0037a0 2024-04-26T19:50:55.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e8031400 con 0x7f83dc0037a0 2024-04-26T19:50:55.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f83f2ffd640 1 -- 172.21.15.79:0/2525076092 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8009650 con 0x7f83dc0037a0 2024-04-26T19:50:55.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fbb0f7fe640 1 -- 172.21.15.79:0/1744001679 learned_addr learned my addr 172.21.15.79:0/1744001679 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fcf3c46d640 1 Processor -- start 2024-04-26T19:50:55.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fbb16207640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb0804ffd0 0x7fbb080d1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fdd82f64640 1 Processor -- start 2024-04-26T19:50:55.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fbb16207640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb080ead00 0x7fbb080e5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fdd82f64640 1 -- start start 2024-04-26T19:50:55.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fcf3c46d640 1 -- start start 2024-04-26T19:50:55.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fbb16207640 1 -- 172.21.15.79:0/1744001679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbb080547c0 con 0x7fbb08006ef0 2024-04-26T19:50:55.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7f83fa4cf640 1 -- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83dc0037a0 msgr2=0x7f83dc003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fbb16207640 1 -- 172.21.15.79:0/1744001679 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb08053fc0 con 0x7fbb0804ffd0 2024-04-26T19:50:55.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7fbb16207640 1 -- 172.21.15.79:0/1744001679 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbb08054940 con 0x7fbb080ead00 2024-04-26T19:50:55.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.653+0000 7f83fa4cf640 1 --2- 172.21.15.79:0/2525076092 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83dc0037a0 0x7f83dc003c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009980 tx=0x7f83dc005830 comp rx=0 tx=0).stop 2024-04-26T19:50:55.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.652+0000 7f99fffff640 1 -- 172.21.15.79:0/1127690119 wait complete. 2024-04-26T19:50:55.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7f99fffff640 1 Processor -- start 2024-04-26T19:50:55.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7f83f8a6c640 1 -- 172.21.15.79:0/2525076092 reap_dead start 2024-04-26T19:50:55.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7feb763da640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb700c4d20 0x7feb70142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbde759e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde81491e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbde759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde8155910 0x7fbde81f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbb0ffff640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb080ead00 0x7fbb080e5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7f83fa4cf640 1 -- 172.21.15.79:0/2525076092 shutdown_connections 2024-04-26T19:50:55.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7f83fa4cf640 1 -- 172.21.15.79:0/2525076092 >> 172.21.15.79:0/2525076092 conn(0x7f83f40b39d0 msgr2=0x7f83f406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbb0effd640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb0804ffd0 0x7fbb080d1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbb0f7fe640 1 -- 172.21.15.79:0/1744001679 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb080ead00 msgr2=0x7fbb080e5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbb0f7fe640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb080ead00 0x7fbb080e5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbb0f7fe640 1 -- 172.21.15.79:0/1744001679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb0804ffd0 msgr2=0x7fbb080d1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbb0f7fe640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb0804ffd0 0x7fbb080d1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbb0f7fe640 1 -- 172.21.15.79:0/1744001679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb080e7520 con 0x7fbb08006ef0 2024-04-26T19:50:55.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7f83fa4cf640 1 -- 172.21.15.79:0/2525076092 shutdown_connections 2024-04-26T19:50:55.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7f45beffd640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45a8004800 0x7f45a8004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f45b8007390 tx=0x7f45a8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24e6a7b832804c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7feb763da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700ce230 0x7feb70072ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7feb763da640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce610 0x7feb70073400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7feb763da640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feb700d1450 con 0x7feb700c4d20 2024-04-26T19:50:55.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fcf3c46d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 0x7fcf340cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb700c4d20 0x7feb70142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbde659c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde81491e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fcf3c46d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf340cad80 0x7fcf34215c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fdd82f64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd7c071850 0x7fdd7c1538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7fbb0f7fe640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08006ef0 0x7fbb080e6fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400ebe0 tx=0x7fbb0400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbde659c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde81491e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44868/0 (socket says 172.21.15.79:44868) 2024-04-26T19:50:55.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3c46d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf340c9740 0x7fcf3420be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3c46d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3408c930 con 0x7fcf340cad80 2024-04-26T19:50:55.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbde659c640 1 -- 172.21.15.79:0/2546646787 learned_addr learned my addr 172.21.15.79:0/2546646787 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3c46d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3408c7b0 con 0x7fcf2000b5b0 2024-04-26T19:50:55.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fdd82f64640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd7c0c6e30 0x7fdd7c15eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7f83fa4cf640 1 -- 172.21.15.79:0/2525076092 wait complete. 2024-04-26T19:50:55.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3c46d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3416efc0 con 0x7fcf340c9740 2024-04-26T19:50:55.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3a209640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 0x7fcf340cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3a209640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 0x7fcf340cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:48414/0 (socket says 172.21.15.79:48414) 2024-04-26T19:50:55.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3a209640 1 -- 172.21.15.79:0/898664997 learned_addr learned my addr 172.21.15.79:0/898664997 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3aa0a640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf340c9740 0x7fcf3420be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbde759e640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbde815b830 0x7fbde806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbde5d9b640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde8155910 0x7fbde81f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb6ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb700c4d20 0x7feb70142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38816/0 (socket says 172.21.15.79:38816) 2024-04-26T19:50:55.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb6ffff640 1 -- 172.21.15.79:0/3934879888 learned_addr learned my addr 172.21.15.79:0/3934879888 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb763da640 1 -- 172.21.15.79:0/3934879888 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feb700c6360 con 0x7feb700ce230 2024-04-26T19:50:55.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb763da640 1 -- 172.21.15.79:0/3934879888 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7feb70071850 con 0x7feb700ce610 2024-04-26T19:50:55.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fdd81f62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd7c071850 0x7fdd7c1538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fdd81f62640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd7c071850 0x7fdd7c1538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:48428/0 (socket says 172.21.15.79:48428) 2024-04-26T19:50:55.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbde815c670 con 0x7fbde815b830 2024-04-26T19:50:55.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbde815c390 con 0x7fbde8155910 2024-04-26T19:50:55.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbafbfff640 1 -- 172.21.15.79:0/1744001679 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04017070 con 0x7fbb08006ef0 2024-04-26T19:50:55.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbafbfff640 1 -- 172.21.15.79:0/1744001679 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0400ecd0 con 0x7fbb08006ef0 2024-04-26T19:50:55.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbde8073c40 con 0x7fbde80b1e00 2024-04-26T19:50:55.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbafbfff640 1 -- 172.21.15.79:0/1744001679 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0400ce70 con 0x7fbb08006ef0 2024-04-26T19:50:55.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fdd81761640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd7c0c6e30 0x7fdd7c15eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fdd81761640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd7c0c6e30 0x7fdd7c15eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44898/0 (socket says 172.21.15.79:44898) 2024-04-26T19:50:55.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fdd81f62640 1 -- 172.21.15.79:0/969404516 learned_addr learned my addr 172.21.15.79:0/969404516 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbb16207640 1 -- 172.21.15.79:0/1744001679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb080e5f40 con 0x7fbb08006ef0 2024-04-26T19:50:55.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fdd82f64640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7c15c790 0x7fdd7c15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.654+0000 7f99fffff640 1 -- start start 2024-04-26T19:50:55.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f99fffff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99f8053c50 0x7f99f804f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f99fffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99f80589d0 0x7f99f80de4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f99fffff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f80e7600 0x7f99f80e63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f99fffff640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f99f8007150 con 0x7f99f80589d0 2024-04-26T19:50:55.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fbb16207640 1 -- 172.21.15.79:0/1744001679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb080e6500 con 0x7fbb08006ef0 2024-04-26T19:50:55.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f99fffff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f99f8006e70 con 0x7f99f8053c50 2024-04-26T19:50:55.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f99fffff640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f99f80061f0 con 0x7f99f80e7600 2024-04-26T19:50:55.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f99fe7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99f80589d0 0x7f99f80de4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f99fe7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99f80589d0 0x7f99f80de4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38834/0 (socket says 172.21.15.79:38834) 2024-04-26T19:50:55.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f99fe7fc640 1 -- 172.21.15.79:0/2415223011 learned_addr learned my addr 172.21.15.79:0/2415223011 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf39a08640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf340cad80 0x7fcf34215c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3aa0a640 1 -- 172.21.15.79:0/898664997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 msgr2=0x7fcf340cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8033070 con 0x7f45a8004800 2024-04-26T19:50:55.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3aa0a640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf2000b5b0 0x7fcf340cc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fdd82763640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7c15c790 0x7fdd7c15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb6f7fe640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700ce230 0x7feb70072ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb74977640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce610 0x7feb70073400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb6ffff640 1 -- 172.21.15.79:0/3934879888 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce610 msgr2=0x7feb70073400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb6ffff640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feb700ce610 0x7feb70073400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb6ffff640 1 -- 172.21.15.79:0/3934879888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700ce230 msgr2=0x7feb70072ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb6ffff640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feb700ce230 0x7feb70072ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7feb6ffff640 1 -- 172.21.15.79:0/3934879888 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb701499b0 con 0x7feb700c4d20 2024-04-26T19:50:55.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7f45be7fc640 1 -- 172.21.15.79:0/4117061758 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b802f670 con 0x7f45a8004800 2024-04-26T19:50:55.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fdd82f64640 1 -- 172.21.15.79:0/969404516 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c071dd0 con 0x7fdd7c15c790 2024-04-26T19:50:55.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7f99fe7fc640 1 -- 172.21.15.79:0/2415223011 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f80e7600 msgr2=0x7f99f80e63d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:50:55.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7f99fe7fc640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f99f80e7600 0x7f99f80e63d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7f99fe7fc640 1 -- 172.21.15.79:0/2415223011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99f8053c50 msgr2=0x7f99f804f260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:50:55.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7f99fe7fc640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f99f8053c50 0x7f99f804f260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7f99fe7fc640 1 -- 172.21.15.79:0/2415223011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f80d19b0 con 0x7f99f80589d0 2024-04-26T19:50:55.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fdd82f64640 1 -- 172.21.15.79:0/969404516 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c071c50 con 0x7fdd7c071850 2024-04-26T19:50:55.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fdd82f64640 1 -- 172.21.15.79:0/969404516 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7c153440 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3aa0a640 1 -- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf340cad80 msgr2=0x7fcf34215c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3aa0a640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf340cad80 0x7fcf34215c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf3aa0a640 1 -- 172.21.15.79:0/898664997 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf340c9ce0 con 0x7fcf340c9740 2024-04-26T19:50:55.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.655+0000 7fcf39a08640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf340cad80 0x7fcf34215c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:55.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fdd81761640 1 -- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd7c071850 msgr2=0x7fdd7c1538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fdd81761640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd7c071850 0x7fdd7c1538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fdd81761640 1 -- 172.21.15.79:0/969404516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7c15c790 msgr2=0x7fdd7c15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fdd81761640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd7c15c790 0x7fdd7c15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fdd81761640 1 -- 172.21.15.79:0/969404516 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c152550 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7f83fa4cf640 1 Processor -- start 2024-04-26T19:50:55.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7f83fa4cf640 1 -- start start 2024-04-26T19:50:55.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7feb6ffff640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb700c4d20 0x7feb70142b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feb600090b0 tx=0x7feb600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fcf3aa0a640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf340c9740 0x7fcf3420be00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcf3408cfc0 tx=0x7fcf2c0051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fdd81761640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd7c0c6e30 0x7fdd7c15eaf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c440 tx=0x7fdd7800c900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7feb6cff9640 1 -- 172.21.15.79:0/3934879888 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60030b80 con 0x7feb700c4d20 2024-04-26T19:50:55.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f45c610a640 1 -- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45a8004800 msgr2=0x7f45a8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f45c610a640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45a8004800 0x7f45a8004c80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f45b8007390 tx=0x7f45a8005830 comp rx=0 tx=0).stop 2024-04-26T19:50:55.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.656+0000 7fbde6d9d640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbde815b830 0x7fbde806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7feb6cff9640 1 -- 172.21.15.79:0/3934879888 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb60030d40 con 0x7feb700c4d20 2024-04-26T19:50:55.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fcf2affd640 1 -- 172.21.15.79:0/898664997 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c02b9e0 con 0x7fcf340c9740 2024-04-26T19:50:55.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7feb763da640 1 -- 172.21.15.79:0/3934879888 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb70149c90 con 0x7feb700c4d20 2024-04-26T19:50:55.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f45c610a640 1 -- 172.21.15.79:0/4117061758 shutdown_connections 2024-04-26T19:50:55.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f45c610a640 1 --2- 172.21.15.79:0/4117061758 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45a8004800 0x7f45a8004c80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f45c610a640 1 -- 172.21.15.79:0/4117061758 >> 172.21.15.79:0/4117061758 conn(0x7f45c00b39d0 msgr2=0x7f45c006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fbde659c640 1 -- 172.21.15.79:0/2546646787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde8155910 msgr2=0x7fbde81f17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fbde659c640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbde8155910 0x7fbde81f17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fbde659c640 1 -- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbde815b830 msgr2=0x7fbde806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fbde659c640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbde815b830 0x7fbde806edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fbde659c640 1 -- 172.21.15.79:0/2546646787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde81f1ce0 con 0x7fbde80b1e00 2024-04-26T19:50:55.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fbde659c640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde81491e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d90 tx=0x7fbde0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7fbdd6ffd640 1 -- 172.21.15.79:0/2546646787 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0040070 con 0x7fbde80b1e00 2024-04-26T19:50:55.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7fbdd6ffd640 1 -- 172.21.15.79:0/2546646787 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde0030ae0 con 0x7fbde80b1e00 2024-04-26T19:50:55.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7fbdd6ffd640 1 -- 172.21.15.79:0/2546646787 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0045460 con 0x7fbde80b1e00 2024-04-26T19:50:55.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde8154bf0 con 0x7fbde80b1e00 2024-04-26T19:50:55.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde81f22f0 con 0x7fbde80b1e00 2024-04-26T19:50:55.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fcf3c46d640 1 -- 172.21.15.79:0/898664997 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf340c9fc0 con 0x7fcf340c9740 2024-04-26T19:50:55.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fcf3c46d640 1 -- 172.21.15.79:0/898664997 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf340ca580 con 0x7fcf340c9740 2024-04-26T19:50:55.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fcf2affd640 1 -- 172.21.15.79:0/898664997 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf2c02bb80 con 0x7fcf340c9740 2024-04-26T19:50:55.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fcf2affd640 1 -- 172.21.15.79:0/898664997 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c034470 con 0x7fcf340c9740 2024-04-26T19:50:55.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7fcf3c46d640 1 -- 172.21.15.79:0/898664997 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf1c001030 con 0x7fcf340c9740 2024-04-26T19:50:55.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fbb16207640 1 -- 172.21.15.79:0/1744001679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbb080052c0 con 0x7fbb08006ef0 2024-04-26T19:50:55.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fdd727fc640 1 -- 172.21.15.79:0/969404516 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78017070 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7fdd727fc640 1 -- 172.21.15.79:0/969404516 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd78005ce0 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7fdd727fc640 1 -- 172.21.15.79:0/969404516 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd780072d0 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7fdd82f64640 1 -- 172.21.15.79:0/969404516 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7c152830 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7fdd82f64640 1 -- 172.21.15.79:0/969404516 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd7c152df0 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f83fa4cf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83dc0037a0 0x7f83f4152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7f83fa4cf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83f40aad40 0x7f83f40ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7f83fa4cf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f415e330 0x7f83f415e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7f83fa4cf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f83f40b0450 con 0x7f83dc0037a0 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7f83fa4cf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f83f4163e70 con 0x7f83f415e330 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7f83fa4cf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f83f40b05d0 con 0x7f83f40aad40 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7f83f37fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83f40aad40 0x7f83f40ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7f83f37fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83f40aad40 0x7f83f40ab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44900/0 (socket says 172.21.15.79:44900) 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.658+0000 7f83f37fe640 1 -- 172.21.15.79:0/1612710444 learned_addr learned my addr 172.21.15.79:0/1612710444 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7f83f8a6c640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f415e330 0x7f83f415e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f99fe7fc640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99f80589d0 0x7f99f80de4f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c7b0 tx=0x7f99f400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f99e77fe640 1 -- 172.21.15.79:0/2415223011 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4016020 con 0x7f99f80589d0 2024-04-26T19:50:55.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f99e77fe640 1 -- 172.21.15.79:0/2415223011 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f4005220 con 0x7f99f80589d0 2024-04-26T19:50:55.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f99e77fe640 1 -- 172.21.15.79:0/2415223011 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f400ce60 con 0x7f99f80589d0 2024-04-26T19:50:55.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f99fffff640 1 -- 172.21.15.79:0/2415223011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f80d1c90 con 0x7f99f80589d0 2024-04-26T19:50:55.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f99fffff640 1 -- 172.21.15.79:0/2415223011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f80d2250 con 0x7f99f80589d0 2024-04-26T19:50:55.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7f99e57fa640 1 -- 172.21.15.79:0/2415223011 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99f8005710 con 0x7f99f80589d0 2024-04-26T19:50:55.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f45c610a640 1 -- 172.21.15.79:0/4117061758 shutdown_connections 2024-04-26T19:50:55.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7f45c610a640 1 -- 172.21.15.79:0/4117061758 wait complete. 2024-04-26T19:50:55.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7f45c610a640 1 Processor -- start 2024-04-26T19:50:55.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7f45c610a640 1 -- start start 2024-04-26T19:50:55.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7f83f8a6c640 1 -- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83f40aad40 msgr2=0x7f83f40ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7f83f8a6c640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f83f40aad40 0x7f83f40ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7f83f8a6c640 1 -- 172.21.15.79:0/1612710444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83dc0037a0 msgr2=0x7f83f4152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7f83f8a6c640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f83dc0037a0 0x7f83f4152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7f83f8a6c640 1 -- 172.21.15.79:0/1612710444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f40aeb10 con 0x7f83f415e330 2024-04-26T19:50:55.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.657+0000 7feb763da640 1 -- 172.21.15.79:0/3934879888 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb701430d0 con 0x7feb700c4d20 2024-04-26T19:50:55.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7feb763da640 1 -- 172.21.15.79:0/3934879888 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb7015c4e0 con 0x7feb700c4d20 2024-04-26T19:50:55.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7feb6cff9640 1 -- 172.21.15.79:0/3934879888 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60044630 con 0x7feb700c4d20 2024-04-26T19:50:55.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7fbafbfff640 1 -- 172.21.15.79:0/1744001679 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb0401a070 con 0x7fbb08006ef0 2024-04-26T19:50:55.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.659+0000 7fbafbfff640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbaf406c9f0 0x7fbaf406eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7fbafbfff640 1 -- 172.21.15.79:0/1744001679 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb0408be20 con 0x7fbb08006ef0 2024-04-26T19:50:55.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7fbafbfff640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbaf40760f0 0x7fbaf4078530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7fbafbfff640 1 -- 172.21.15.79:0/1744001679 --> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbb04022e50 con 0x7fbaf40760f0 2024-04-26T19:50:55.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7feb6cff9640 1 -- 172.21.15.79:0/3934879888 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb6004d430 con 0x7feb700c4d20 2024-04-26T19:50:55.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7fbafbfff640 1 -- 172.21.15.79:0/1744001679 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbb04055a00 con 0x7fbb08006ef0 2024-04-26T19:50:55.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7f83f8a6c640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f415e330 0x7f83f415e7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f83f406d550 tx=0x7f83ec00cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7f45c610a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45a8004800 0x7f45c00af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7f45c610a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45c00737b0 0x7f45c01fd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7f45c610a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45c01590c0 0x7f45c00abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7f45c610a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b05d0 con 0x7f45c00737b0 2024-04-26T19:50:55.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7f45c610a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b0450 con 0x7f45a8004800 2024-04-26T19:50:55.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7f45c610a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f45c015e1c0 con 0x7f45c01590c0 2024-04-26T19:50:55.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.660+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbde8050f10 con 0x7fbde80b1e00 2024-04-26T19:50:55.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7fbdd6ffd640 1 -- 172.21.15.79:0/2546646787 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbde0039040 con 0x7fbde80b1e00 2024-04-26T19:50:55.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7fbdd6ffd640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdc806c9c0 0x7fbdc806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7feb6cff9640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feb5c06c940 0x7feb5c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7feb6f7fe640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feb5c06c940 0x7feb5c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7feb6cff9640 1 -- 172.21.15.79:0/3934879888 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7feb600acfb0 con 0x7feb700c4d20 2024-04-26T19:50:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7feb6cff9640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7feb5c0760c0 0x7feb5c078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7feb6cff9640 1 -- 172.21.15.79:0/3934879888 --> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb5c078bd0 con 0x7feb5c0760c0 2024-04-26T19:50:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7feb6cff9640 1 -- 172.21.15.79:0/3934879888 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7feb600ad490 con 0x7feb700c4d20 2024-04-26T19:50:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7feb6f7fe640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feb5c06c940 0x7feb5c06ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb7014f9f0 tx=0x7feb64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7fcf2affd640 1 -- 172.21.15.79:0/898664997 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf2c03c070 con 0x7fcf340c9740 2024-04-26T19:50:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7fcf2affd640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf1806c9c0 0x7fcf1806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7fcf2affd640 1 -- 172.21.15.79:0/898664997 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcf2c0ab650 con 0x7fcf340c9740 2024-04-26T19:50:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7fcf3a209640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf1806c9c0 0x7fcf1806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7fdd5ffff640 1 -- 172.21.15.79:0/969404516 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd7c050f10 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7fdd727fc640 1 -- 172.21.15.79:0/969404516 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd7801a070 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7fdd727fc640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c9f0 0x7fdd6006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7f83f0ff9640 1 -- 172.21.15.79:0/1612710444 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec017070 con 0x7f83f415e330 2024-04-26T19:50:55.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7f83f0ff9640 1 -- 172.21.15.79:0/1612710444 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83ec00ce70 con 0x7f83f415e330 2024-04-26T19:50:55.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f83f0ff9640 1 -- 172.21.15.79:0/1612710444 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4007b40 con 0x7f83f415e330 2024-04-26T19:50:55.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f40aedf0 con 0x7f83f415e330 2024-04-26T19:50:55.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f40af3b0 con 0x7f83f415e330 2024-04-26T19:50:55.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f99e77fe640 1 -- 172.21.15.79:0/2415223011 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99f4007030 con 0x7f99f80589d0 2024-04-26T19:50:55.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f99e77fe640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f99ec06c970 0x7f99ec06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f99e77fe640 1 -- 172.21.15.79:0/2415223011 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f99f408c430 con 0x7f99f80589d0 2024-04-26T19:50:55.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f99feffd640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f99ec06c970 0x7f99ec06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7f45bf7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45a8004800 0x7f45c00af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7f45bffff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45c01590c0 0x7f45c00abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7f45bffff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45c01590c0 0x7f45c00abc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44908/0 (socket says 172.21.15.79:44908) 2024-04-26T19:50:55.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7f45bffff640 1 -- 172.21.15.79:0/1724023253 learned_addr learned my addr 172.21.15.79:0/1724023253 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7f45bffff640 1 -- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45a8004800 msgr2=0x7f45c00af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f45bffff640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45a8004800 0x7f45c00af2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f45bffff640 1 -- 172.21.15.79:0/1724023253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45c00737b0 msgr2=0x7f45c01fd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f45bffff640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f45c00737b0 0x7f45c01fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f45bffff640 1 -- 172.21.15.79:0/1724023253 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c00ae200 con 0x7f45c01590c0 2024-04-26T19:50:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f45bffff640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45c01590c0 0x7f45c00abc30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f45c00b0370 tx=0x7f45b400e960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7feb74977640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7feb5c0760c0 0x7feb5c078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f99e77fe640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7f99ec0760f0 0x7f99ec078530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f99e77fe640 1 -- 172.21.15.79:0/2415223011 --> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99ec078c00 con 0x7f99ec0760f0 2024-04-26T19:50:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7f99e77fe640 1 -- 172.21.15.79:0/2415223011 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99f408c7e0 con 0x7f99f80589d0 2024-04-26T19:50:55.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fdd81f62640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c9f0 0x7fdd6006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fdd727fc640 1 -- 172.21.15.79:0/969404516 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdd7808af40 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7fbdd6ffd640 1 -- 172.21.15.79:0/2546646787 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbde00ad910 con 0x7fbde80b1e00 2024-04-26T19:50:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7f99ff7fe640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7f99ec0760f0 0x7f99ec078530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fbdd6ffd640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7fbdc8076140 0x7fbdc8078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fcf2affd640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fcf18076140 0x7fcf18078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fdd727fc640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7fdd60076160 0x7fdd600785a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fdd727fc640 1 -- 172.21.15.79:0/969404516 --> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd78005ce0 con 0x7fdd60076160 2024-04-26T19:50:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fcf2affd640 1 -- 172.21.15.79:0/898664997 --> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf18078c50 con 0x7fcf18076140 2024-04-26T19:50:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fdd727fc640 1 -- 172.21.15.79:0/969404516 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdd7808d5a0 con 0x7fdd7c0c6e30 2024-04-26T19:50:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fcf2affd640 1 -- 172.21.15.79:0/898664997 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf2c0b2050 con 0x7fcf340c9740 2024-04-26T19:50:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fdd82763640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7fdd60076160 0x7fdd600785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7fbb0effd640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbaf406c9f0 0x7fbaf406eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.661+0000 7fbb0effd640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbaf406c9f0 0x7fbaf406eeb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb08039f70 tx=0x7fbafc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.662+0000 7fbb0ffff640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbaf40760f0 0x7fbaf4078530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fbb0ffff640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbaf40760f0 0x7fbaf4078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fbde6d9d640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7fbdc8076140 0x7fbdc8078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fbde5d9b640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdc806c9c0 0x7fbdc806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7fbde6d9d640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7fbdc8076140 0x7fbdc8078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7fbdd6ffd640 1 -- 172.21.15.79:0/2546646787 --> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbde0032e70 con 0x7fbdc8076140 2024-04-26T19:50:55.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7f99ff7fe640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7f99ec0760f0 0x7f99ec078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7fcf39a08640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fcf18076140 0x7fcf18078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7fbdd6ffd640 1 -- 172.21.15.79:0/2546646787 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbde00add00 con 0x7fbde80b1e00 2024-04-26T19:50:55.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7fcf3a209640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf1806c9c0 0x7fcf1806ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000b410 tx=0x7fcf30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7fdd81f62640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c9f0 0x7fdd6006eeb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009ba0 tx=0x7fdd6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7fbafbfff640 1 -- 172.21.15.79:0/1744001679 <== osd.3 v2:172.21.15.79:6816/1557374359 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbb04022e50 con 0x7fbaf40760f0 2024-04-26T19:50:55.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7fdd82763640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7fdd60076160 0x7fdd600785a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7fbde5d9b640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdc806c9c0 0x7fbdc806ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbde81508f0 tx=0x7fbddc009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7feb74977640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7feb5c0760c0 0x7feb5c078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.663+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83f40506c0 con 0x7f83f415e330 2024-04-26T19:50:55.759 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7f83f0ff9640 1 -- 172.21.15.79:0/1612710444 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f83ec003970 con 0x7f83f415e330 2024-04-26T19:50:55.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7fcf39a08640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fcf18076140 0x7fcf18078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7f99e77fe640 1 -- 172.21.15.79:0/2415223011 <== osd.6 v2:172.21.15.83:6800/2962195076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99ec078c00 con 0x7f99ec0760f0 2024-04-26T19:50:55.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7f45a7fff640 1 -- 172.21.15.79:0/1724023253 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b4019070 con 0x7f45c01590c0 2024-04-26T19:50:55.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7f45a7fff640 1 -- 172.21.15.79:0/1724023253 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b40105d0 con 0x7f45c01590c0 2024-04-26T19:50:55.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7f45a7fff640 1 -- 172.21.15.79:0/1724023253 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b40109a0 con 0x7f45c01590c0 2024-04-26T19:50:55.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.665+0000 7feb6cff9640 1 -- 172.21.15.79:0/3934879888 <== osd.1 v2:172.21.15.79:6824/2915211239 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feb5c078bd0 con 0x7feb5c0760c0 2024-04-26T19:50:55.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7f45c610a640 1 -- 172.21.15.79:0/1724023253 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c00ae4e0 con 0x7f45c01590c0 2024-04-26T19:50:55.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.664+0000 7f45c610a640 1 -- 172.21.15.79:0/1724023253 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c00aeaa0 con 0x7f45c01590c0 2024-04-26T19:50:55.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.665+0000 7fbdd6ffd640 1 -- 172.21.15.79:0/2546646787 <== osd.0 v2:172.21.15.79:6800/2567076951 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbde0032e70 con 0x7fbdc8076140 2024-04-26T19:50:55.762 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.665+0000 7f99feffd640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f99ec06c970 0x7f99ec06ee30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f99f000b7f0 tx=0x7f99f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.665+0000 7f83f0ff9640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f83e006c9f0 0x7f83e006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.665+0000 7f83f0ff9640 1 -- 172.21.15.79:0/1612710444 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f83ec08af30 con 0x7f83f415e330 2024-04-26T19:50:55.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.665+0000 7f83f0ff9640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f83e0076170 0x7f83e00785b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.665+0000 7f83f0ff9640 1 -- 172.21.15.79:0/1612710444 --> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83e0078c80 con 0x7f83e0076170 2024-04-26T19:50:55.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.665+0000 7f83f0ff9640 1 -- 172.21.15.79:0/1612710444 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f83ec08d590 con 0x7f83f415e330 2024-04-26T19:50:55.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.665+0000 7f83f37fe640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f83e0076170 0x7f83e00785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.666+0000 7fdd727fc640 1 -- 172.21.15.79:0/969404516 <== osd.4 v2:172.21.15.83:6808/3328215207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd78005ce0 con 0x7fdd60076160 2024-04-26T19:50:55.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.666+0000 7fcf2affd640 1 -- 172.21.15.79:0/898664997 <== osd.2 v2:172.21.15.79:6808/569648363 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf18078c50 con 0x7fcf18076140 2024-04-26T19:50:55.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.666+0000 7f45a5ffb640 1 -- 172.21.15.79:0/1724023253 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f45c00506c0 con 0x7f45c01590c0 2024-04-26T19:50:55.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.666+0000 7f83f37fe640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f83e0076170 0x7f83e00785b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.666+0000 7f83f3fff640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f83e006c9f0 0x7f83e006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.667+0000 7f83f3fff640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f83e006c9f0 0x7f83e006eeb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f83e4001590 tx=0x7f83e4000f10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.667+0000 7f83f0ff9640 1 -- 172.21.15.79:0/1612710444 <== osd.5 v2:172.21.15.83:6824/3851137838 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83e0078c80 con 0x7f83e0076170 2024-04-26T19:50:55.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.667+0000 7f45a7fff640 1 -- 172.21.15.79:0/1724023253 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45b401c070 con 0x7f45c01590c0 2024-04-26T19:50:55.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.667+0000 7f45a7fff640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f45ac06c970 0x7f45ac06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.668+0000 7f45a7fff640 1 -- 172.21.15.79:0/1724023253 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f45b4056ef0 con 0x7f45c01590c0 2024-04-26T19:50:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.668+0000 7f45a7fff640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7f45ac076d40 0x7f45ac0791e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.668+0000 7f45a7fff640 1 -- 172.21.15.79:0/1724023253 --> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f45ac0798b0 con 0x7f45ac076d40 2024-04-26T19:50:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.668+0000 7f45bf7fe640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f45ac06c970 0x7f45ac06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.668+0000 7f45a7fff640 1 -- 172.21.15.79:0/1724023253 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f45b4055880 con 0x7f45c01590c0 2024-04-26T19:50:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.669+0000 7f45bf7fe640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f45ac06c970 0x7f45ac06ee30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45c0050be0 tx=0x7f45b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.670+0000 7f45beffd640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7f45ac076d40 0x7f45ac0791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.673+0000 7f45beffd640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7f45ac076d40 0x7f45ac0791e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:55.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.674+0000 7f45a7fff640 1 -- 172.21.15.79:0/1724023253 <== osd.7 v2:172.21.15.83:6816/4292059702 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f45ac0798b0 con 0x7f45ac076d40 2024-04-26T19:50:55.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.686+0000 7feb763da640 1 -- 172.21.15.79:0/3934879888 --> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb700506c0 con 0x7feb5c0760c0 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.687+0000 7feb6cff9640 1 -- 172.21.15.79:0/3934879888 <== osd.1 v2:172.21.15.79:6824/2915211239 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feb700506c0 con 0x7feb5c0760c0 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.688+0000 7feb567fc640 1 -- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7feb5c0760c0 msgr2=0x7feb5c078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.688+0000 7feb567fc640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7feb5c0760c0 0x7feb5c078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.688+0000 7feb567fc640 1 -- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feb5c06c940 msgr2=0x7feb5c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.688+0000 7feb567fc640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7feb5c06c940 0x7feb5c06ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7feb7014f9f0 tx=0x7feb64008040 comp rx=0 tx=0).stop 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.688+0000 7feb567fc640 1 -- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb700c4d20 msgr2=0x7feb70142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.688+0000 7feb567fc640 1 --2- 172.21.15.79:0/3934879888 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feb700c4d20 0x7feb70142b90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7feb600090b0 tx=0x7feb600046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.688+0000 7feb74977640 1 -- 172.21.15.79:0/3934879888 reap_dead start 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.688+0000 7feb567fc640 1 -- 172.21.15.79:0/3934879888 shutdown_connections 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.688+0000 7feb567fc640 1 -- 172.21.15.79:0/3934879888 >> 172.21.15.79:0/3934879888 conn(0x7feb7006f580 msgr2=0x7feb7015ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.689+0000 7feb567fc640 1 -- 172.21.15.79:0/3934879888 shutdown_connections 2024-04-26T19:50:55.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.689+0000 7feb567fc640 1 -- 172.21.15.79:0/3934879888 wait complete. 2024-04-26T19:50:55.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.696+0000 7fcf3c46d640 1 -- 172.21.15.79:0/898664997 --> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcf1c002dd0 con 0x7fcf18076140 2024-04-26T19:50:55.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.697+0000 7fcf2affd640 1 -- 172.21.15.79:0/898664997 <== osd.2 v2:172.21.15.79:6808/569648363 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcf1c002dd0 con 0x7fcf18076140 2024-04-26T19:50:55.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fcf28ff9640 1 -- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fcf18076140 msgr2=0x7fcf18078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fcf28ff9640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fcf18076140 0x7fcf18078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fcf28ff9640 1 -- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf1806c9c0 msgr2=0x7fcf1806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fcf28ff9640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf1806c9c0 0x7fcf1806ee80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000b410 tx=0x7fcf30008040 comp rx=0 tx=0).stop 2024-04-26T19:50:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fcf28ff9640 1 -- 172.21.15.79:0/898664997 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf340c9740 msgr2=0x7fcf3420be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fcf28ff9640 1 --2- 172.21.15.79:0/898664997 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf340c9740 0x7fcf3420be00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fcf3408cfc0 tx=0x7fcf2c0051f0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.696+0000 7fbb16207640 1 -- 172.21.15.79:0/1744001679 --> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbb080e6c10 con 0x7fbaf40760f0 2024-04-26T19:50:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.697+0000 7fbafbfff640 1 -- 172.21.15.79:0/1744001679 <== osd.3 v2:172.21.15.79:6816/1557374359 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbb080e6c10 con 0x7fbaf40760f0 2024-04-26T19:50:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 -- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbaf40760f0 msgr2=0x7fbaf4078530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbaf40760f0 0x7fbaf4078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 -- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbaf406c9f0 msgr2=0x7fbaf406eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.772 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbaf406c9f0 0x7fbaf406eeb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb08039f70 tx=0x7fbafc009040 comp rx=0 tx=0).stop 2024-04-26T19:50:55.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 -- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08006ef0 msgr2=0x7fbb080e6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 --2- 172.21.15.79:0/1744001679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb08006ef0 0x7fbb080e6fe0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400ebe0 tx=0x7fbb0400c520 comp rx=0 tx=0).stop 2024-04-26T19:50:55.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbb0ffff640 1 -- 172.21.15.79:0/1744001679 reap_dead start 2024-04-26T19:50:55.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7f99e57fa640 1 -- 172.21.15.79:0/2415223011 --> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f99f800e100 con 0x7f99ec0760f0 2024-04-26T19:50:55.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 --> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbde8154880 con 0x7fbdc8076140 2024-04-26T19:50:55.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fcf3aa0a640 1 -- 172.21.15.79:0/898664997 reap_dead start 2024-04-26T19:50:55.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fcf28ff9640 1 -- 172.21.15.79:0/898664997 shutdown_connections 2024-04-26T19:50:55.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fcf28ff9640 1 -- 172.21.15.79:0/898664997 >> 172.21.15.79:0/898664997 conn(0x7fcf340b4e80 msgr2=0x7fcf3406f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 -- 172.21.15.79:0/1744001679 shutdown_connections 2024-04-26T19:50:55.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 -- 172.21.15.79:0/1744001679 >> 172.21.15.79:0/1744001679 conn(0x7fbb08009fb0 msgr2=0x7fbb08036b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7fdd5ffff640 1 -- 172.21.15.79:0/969404516 --> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd7c15b7e0 con 0x7fdd60076160 2024-04-26T19:50:55.775 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99e77fe640 1 -- 172.21.15.79:0/2415223011 <== osd.6 v2:172.21.15.83:6800/2962195076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f99f800e100 con 0x7f99ec0760f0 2024-04-26T19:50:55.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 -- 172.21.15.79:0/1744001679 shutdown_connections 2024-04-26T19:50:55.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.698+0000 7fbaf9ffb640 1 -- 172.21.15.79:0/1744001679 wait complete. 2024-04-26T19:50:55.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7fbdd6ffd640 1 -- 172.21.15.79:0/2546646787 <== osd.0 v2:172.21.15.79:6800/2567076951 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fbde8154880 con 0x7fbdc8076140 2024-04-26T19:50:55.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7fcf28ff9640 1 -- 172.21.15.79:0/898664997 shutdown_connections 2024-04-26T19:50:55.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7fcf28ff9640 1 -- 172.21.15.79:0/898664997 wait complete. 2024-04-26T19:50:55.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99fffff640 1 -- 172.21.15.79:0/2415223011 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7f99ec0760f0 msgr2=0x7f99ec078530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99fffff640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7f99ec0760f0 0x7f99ec078530 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99fffff640 1 -- 172.21.15.79:0/2415223011 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f99ec06c970 msgr2=0x7f99ec06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99fffff640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f99ec06c970 0x7f99ec06ee30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f99f000b7f0 tx=0x7f99f0008040 comp rx=0 tx=0).stop 2024-04-26T19:50:55.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99fffff640 1 -- 172.21.15.79:0/2415223011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99f80589d0 msgr2=0x7f99f80de4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99fffff640 1 --2- 172.21.15.79:0/2415223011 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f99f80589d0 0x7f99f80de4f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c7b0 tx=0x7f99f400cc70 comp rx=0 tx=0).stop 2024-04-26T19:50:55.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99ff7fe640 1 -- 172.21.15.79:0/2415223011 reap_dead start 2024-04-26T19:50:55.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99fffff640 1 -- 172.21.15.79:0/2415223011 shutdown_connections 2024-04-26T19:50:55.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99fffff640 1 -- 172.21.15.79:0/2415223011 >> 172.21.15.79:0/2415223011 conn(0x7f99f8058ef0 msgr2=0x7f99f80e3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.699+0000 7f99fffff640 1 -- 172.21.15.79:0/2415223011 shutdown_connections 2024-04-26T19:50:55.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7f99fffff640 1 -- 172.21.15.79:0/2415223011 wait complete. 2024-04-26T19:50:55.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7fbdc8076140 msgr2=0x7fbdc8078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fbde759e640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7fbdc8076140 0x7fbdc8078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdc806c9c0 msgr2=0x7fbdc806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fbde759e640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbdc806c9c0 0x7fbdc806ee80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbde81508f0 tx=0x7fbddc009060 comp rx=0 tx=0).stop 2024-04-26T19:50:55.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 msgr2=0x7fbde81491e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fbde759e640 1 --2- 172.21.15.79:0/2546646787 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbde80b1e00 0x7fbde81491e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbde0009d90 tx=0x7fbde0004a40 comp rx=0 tx=0).stop 2024-04-26T19:50:55.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fbde6d9d640 1 -- 172.21.15.79:0/2546646787 reap_dead start 2024-04-26T19:50:55.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fdd727fc640 1 -- 172.21.15.79:0/969404516 <== osd.4 v2:172.21.15.83:6808/3328215207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fdd7c15b7e0 con 0x7fdd60076160 2024-04-26T19:50:55.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 shutdown_connections 2024-04-26T19:50:55.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.700+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 >> 172.21.15.79:0/2546646787 conn(0x7fbde80b4e80 msgr2=0x7fbde80b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 shutdown_connections 2024-04-26T19:50:55.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fbde759e640 1 -- 172.21.15.79:0/2546646787 wait complete. 2024-04-26T19:50:55.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fdd5ffff640 1 -- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7fdd60076160 msgr2=0x7fdd600785a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fdd5ffff640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7fdd60076160 0x7fdd600785a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fdd5ffff640 1 -- 172.21.15.79:0/969404516 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c9f0 msgr2=0x7fdd6006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fdd5ffff640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c9f0 0x7fdd6006eeb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009ba0 tx=0x7fdd6c008040 comp rx=0 tx=0).stop 2024-04-26T19:50:55.781 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fdd5ffff640 1 -- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd7c0c6e30 msgr2=0x7fdd7c15eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fdd5ffff640 1 --2- 172.21.15.79:0/969404516 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd7c0c6e30 0x7fdd7c15eaf0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800c440 tx=0x7fdd7800c900 comp rx=0 tx=0).stop 2024-04-26T19:50:55.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fdd82763640 1 -- 172.21.15.79:0/969404516 reap_dead start 2024-04-26T19:50:55.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.701+0000 7fdd5ffff640 1 -- 172.21.15.79:0/969404516 shutdown_connections 2024-04-26T19:50:55.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.702+0000 7fdd5ffff640 1 -- 172.21.15.79:0/969404516 >> 172.21.15.79:0/969404516 conn(0x7fdd7c06eb00 msgr2=0x7fdd7c0740f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.702+0000 7fdd5ffff640 1 -- 172.21.15.79:0/969404516 shutdown_connections 2024-04-26T19:50:55.782 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.702+0000 7fdd5ffff640 1 -- 172.21.15.79:0/969404516 wait complete. 2024-04-26T19:50:55.783 INFO:teuthology.orchestra.run.smithi079.stdout:55834574860 2024-04-26T19:50:55.783 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T19:50:55.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.705+0000 7f45a5ffb640 1 -- 172.21.15.79:0/1724023253 --> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f45c00d0720 con 0x7f45ac076d40 2024-04-26T19:50:55.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.705+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 --> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f83f40d0720 con 0x7f83e0076170 2024-04-26T19:50:55.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.706+0000 7f45a7fff640 1 -- 172.21.15.79:0/1724023253 <== osd.7 v2:172.21.15.83:6816/4292059702 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f45c00d0720 con 0x7f45ac076d40 2024-04-26T19:50:55.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.706+0000 7f83f0ff9640 1 -- 172.21.15.79:0/1612710444 <== osd.5 v2:172.21.15.83:6824/3851137838 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f83f40d0720 con 0x7f83e0076170 2024-04-26T19:50:55.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.706+0000 7f45a5ffb640 1 -- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7f45ac076d40 msgr2=0x7f45ac0791e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.706+0000 7f45a5ffb640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7f45ac076d40 0x7f45ac0791e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.706+0000 7f45a5ffb640 1 -- 172.21.15.79:0/1724023253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f45ac06c970 msgr2=0x7f45ac06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.706+0000 7f45a5ffb640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f45ac06c970 0x7f45ac06ee30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f45c0050be0 tx=0x7f45b0008040 comp rx=0 tx=0).stop 2024-04-26T19:50:55.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.706+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f83e0076170 msgr2=0x7f83e00785b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.706+0000 7f83fa4cf640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f83e0076170 0x7f83e00785b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:55.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.706+0000 7f45a5ffb640 1 -- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45c01590c0 msgr2=0x7f45c00abc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f45a5ffb640 1 --2- 172.21.15.79:0/1724023253 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45c01590c0 0x7f45c00abc30 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f45c00b0370 tx=0x7f45b400e960 comp rx=0 tx=0).stop 2024-04-26T19:50:55.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f45bffff640 1 -- 172.21.15.79:0/1724023253 reap_dead start 2024-04-26T19:50:55.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f45a5ffb640 1 -- 172.21.15.79:0/1724023253 shutdown_connections 2024-04-26T19:50:55.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f45a5ffb640 1 -- 172.21.15.79:0/1724023253 >> 172.21.15.79:0/1724023253 conn(0x7f45c00b39d0 msgr2=0x7f45c00b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f83e006c9f0 msgr2=0x7f83e006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f83fa4cf640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f83e006c9f0 0x7f83e006eeb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f83e4001590 tx=0x7f83e4000f10 comp rx=0 tx=0).stop 2024-04-26T19:50:55.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f415e330 msgr2=0x7f83f415e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f83fa4cf640 1 --2- 172.21.15.79:0/1612710444 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f83f415e330 0x7f83f415e7b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f83f406d550 tx=0x7f83ec00cc20 comp rx=0 tx=0).stop 2024-04-26T19:50:55.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f83f8a6c640 1 -- 172.21.15.79:0/1612710444 reap_dead start 2024-04-26T19:50:55.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f45a5ffb640 1 -- 172.21.15.79:0/1724023253 shutdown_connections 2024-04-26T19:50:55.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 shutdown_connections 2024-04-26T19:50:55.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f45a5ffb640 1 -- 172.21.15.79:0/1724023253 wait complete. 2024-04-26T19:50:55.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 >> 172.21.15.79:0/1612710444 conn(0x7f83f40b39d0 msgr2=0x7f83f40717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:55.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 shutdown_connections 2024-04-26T19:50:55.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.707+0000 7f83fa4cf640 1 -- 172.21.15.79:0/1612710444 wait complete. 2024-04-26T19:50:55.791 INFO:teuthology.orchestra.run.smithi079.stdout:55834574859 2024-04-26T19:50:55.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T19:50:55.792 INFO:teuthology.orchestra.run.smithi079.stdout:55834574857 2024-04-26T19:50:55.793 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T19:50:55.794 INFO:teuthology.orchestra.run.smithi079.stdout:55834574859 2024-04-26T19:50:55.794 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T19:50:55.795 INFO:teuthology.orchestra.run.smithi079.stdout:55834574858 2024-04-26T19:50:55.795 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T19:50:55.797 INFO:teuthology.orchestra.run.smithi079.stdout:55834574860 2024-04-26T19:50:55.797 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T19:50:55.798 INFO:teuthology.orchestra.run.smithi079.stdout:55834574858 2024-04-26T19:50:55.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T19:50:55.800 INFO:teuthology.orchestra.run.smithi079.stdout:55834574858 2024-04-26T19:50:55.800 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T19:50:55.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.993+0000 7f356759e640 1 Processor -- start 2024-04-26T19:50:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.994+0000 7f150e463640 1 Processor -- start 2024-04-26T19:50:55.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.994+0000 7f150e463640 1 -- start start 2024-04-26T19:50:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f150e463640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f150000c0a0 0x7f1500038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f150e463640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1500033ef0 0x7f150004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f150e463640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15000544c0 con 0x7f1500053c70 2024-04-26T19:50:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f150e463640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f15000541e0 con 0x7f1500033ef0 2024-04-26T19:50:55.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f150e463640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1500005f80 con 0x7f150000c0a0 2024-04-26T19:50:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.996+0000 7f150dc62640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1500033ef0 0x7f150004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.996+0000 7f150dc62640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1500033ef0 0x7f150004fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38878/0 (socket says 172.21.15.79:38878) 2024-04-26T19:50:55.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.996+0000 7f150dc62640 1 -- 172.21.15.79:0/2601159887 learned_addr learned my addr 172.21.15.79:0/2601159887 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.994+0000 7f356759e640 1 -- start start 2024-04-26T19:50:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f356759e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35600069a0 0x7f3560050540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f356759e640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3560050c20 con 0x7f35600e75e0 2024-04-26T19:50:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f356759e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3560050f00 con 0x7f35600345d0 2024-04-26T19:50:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f356759e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f35600511e0 con 0x7f35600069a0 2024-04-26T19:50:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f3565d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35600069a0 0x7f3560050540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.996+0000 7f3565d9b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35600069a0 0x7f3560050540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:48476/0 (socket says 172.21.15.79:48476) 2024-04-26T19:50:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.996+0000 7f3565d9b640 1 -- 172.21.15.79:0/1080092447 learned_addr learned my addr 172.21.15.79:0/1080092447 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.996+0000 7f3565d9b640 1 -- 172.21.15.79:0/1080092447 >> v1:172.21.15.83:6789/0 conn(0x7f35600345d0 legacy=0x7f35600534e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T19:50:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.996+0000 7f3565d9b640 1 -- 172.21.15.79:0/1080092447 >> v1:172.21.15.79:6789/0 conn(0x7f35600e75e0 legacy=0x7f3560006290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.996+0000 7f3565d9b640 1 -- 172.21.15.79:0/1080092447 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35600514c0 con 0x7f35600069a0 2024-04-26T19:50:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f150d461640 1 --2- 172.21.15.79:0/2601159887 >> v2:172.21.15.83:3300/0 conn(0x7f150000c0a0 0x7f1500038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f010675f640 1 Processor -- start 2024-04-26T19:50:55.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f010675f640 1 -- start start 2024-04-26T19:50:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f150dc62640 1 -- 172.21.15.79:0/2601159887 >> v2:172.21.15.83:3300/0 conn(0x7f150000c0a0 msgr2=0x7f1500038080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f150dc62640 1 --2- 172.21.15.79:0/2601159887 >> v2:172.21.15.83:3300/0 conn(0x7f150000c0a0 0x7f1500038080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f150dc62640 1 -- 172.21.15.79:0/2601159887 >> v1:172.21.15.79:6789/0 conn(0x7f1500053c70 legacy=0x7f15000df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f150dc62640 1 -- 172.21.15.79:0/2601159887 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1500006260 con 0x7f1500033ef0 2024-04-26T19:50:56.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7fcf9b59e640 1 Processor -- start 2024-04-26T19:50:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.995+0000 7f4b2ab24640 1 Processor -- start 2024-04-26T19:50:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.996+0000 7f4b2ab24640 1 -- start start 2024-04-26T19:50:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f4b2ab24640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b24073ee0 0x7f4b2408ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f4b2ab24640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4b2416c3a0 0x7f4b24071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f010675f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f00f80ead00 0x7f00f804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f010675f640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f00f804ffd0 0x7f00f8037bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f010675f640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00f8053720 con 0x7f00f8038110 2024-04-26T19:50:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f010675f640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f00f8054330 con 0x7f00f804ffd0 2024-04-26T19:50:56.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f010675f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f00f8053a00 con 0x7f00f80ead00 2024-04-26T19:50:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f4b2ab24640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b24071f70 con 0x7f4b2416bfc0 2024-04-26T19:50:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f4b2ab24640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f4b24071d50 con 0x7f4b2416c3a0 2024-04-26T19:50:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f4b2ab24640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4b24072250 con 0x7f4b24073ee0 2024-04-26T19:50:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f4b288c0640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4b2416bfc0 0x7f4b240737d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46324/0 (socket says 172.21.15.79:46324) 2024-04-26T19:50:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f4b288c0640 1 -- 172.21.15.79:0/63947449 learned_addr learned my addr 172.21.15.79:0/63947449 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:56.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.997+0000 7f4b23fff640 1 --2- 172.21.15.79:0/63947449 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b24073ee0 0x7f4b2408ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f0105f5e640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f00f8038110 0x7f00f8038580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:46332/0 (socket says 172.21.15.79:46332) 2024-04-26T19:50:56.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f0105f5e640 1 -- 172.21.15.79:0/4239015491 learned_addr learned my addr 172.21.15.79:0/4239015491 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:56.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f4b290c1640 1 --2- 172.21.15.79:0/63947449 >> v2:172.21.15.79:3300/0 conn(0x7f4b2416c3a0 0x7f4b24071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f0104f5c640 1 --2- 172.21.15.79:0/4239015491 >> v2:172.21.15.79:3300/0 conn(0x7f00f804ffd0 0x7f00f8037bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f4b237fe640 1 -- 172.21.15.79:0/63947449 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1151099427 0 0) 0x7f4b24071f70 con 0x7f4b2416bfc0 2024-04-26T19:50:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7f4b237fe640 1 -- 172.21.15.79:0/63947449 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b0c003500 con 0x7f4b2416bfc0 2024-04-26T19:50:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f4b23fff640 1 -- 172.21.15.79:0/63947449 >> v2:172.21.15.79:3300/0 conn(0x7f4b2416c3a0 msgr2=0x7f4b24071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f4b23fff640 1 --2- 172.21.15.79:0/63947449 >> v2:172.21.15.79:3300/0 conn(0x7f4b2416c3a0 0x7f4b24071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f4b23fff640 1 -- 172.21.15.79:0/63947449 >> v1:172.21.15.79:6789/0 conn(0x7f4b2416bfc0 legacy=0x7f4b240737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.007 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f4b23fff640 1 -- 172.21.15.79:0/63947449 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b24072530 con 0x7f4b24073ee0 2024-04-26T19:50:56.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f0104f5c640 1 -- 172.21.15.79:0/4239015491 >> v2:172.21.15.83:3300/0 conn(0x7f00f80ead00 msgr2=0x7f00f804fa90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T19:50:56.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f0104f5c640 1 --2- 172.21.15.79:0/4239015491 >> v2:172.21.15.83:3300/0 conn(0x7f00f80ead00 0x7f00f804fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f00f7fff640 1 -- 172.21.15.79:0/4239015491 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1546615627 0 0) 0x7f00f8053720 con 0x7f00f8038110 2024-04-26T19:50:56.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f0104f5c640 1 -- 172.21.15.79:0/4239015491 >> v1:172.21.15.79:6789/0 conn(0x7f00f8038110 legacy=0x7f00f8038580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f0104f5c640 1 -- 172.21.15.79:0/4239015491 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f8053ce0 con 0x7f00f804ffd0 2024-04-26T19:50:56.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7fd3874cc640 1 Processor -- start 2024-04-26T19:50:56.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.998+0000 7fcf9b59e640 1 -- start start 2024-04-26T19:50:56.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7f3565d9b640 1 --2- 172.21.15.79:0/1080092447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600069a0 0x7f3560050540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f356806b410 tx=0x7f35680b7c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a7ad75383eb437 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7fcf9b59e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c1678e0 0x7fcf9c164da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7fcf9b59e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcf9c14d060 0x7fcf9c0cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7fcf9b59e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf9c0d0800 con 0x7fcf9c14cc80 2024-04-26T19:50:56.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fe25b159640 1 Processor -- start 2024-04-26T19:50:56.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7fcf9b59e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcf9c071a20 con 0x7fcf9c14d060 2024-04-26T19:50:56.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7f356559a640 1 -- 172.21.15.79:0/1080092447 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35680b8a50 con 0x7f35600069a0 2024-04-26T19:50:56.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7f356559a640 1 -- 172.21.15.79:0/1080092447 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35680bc070 con 0x7f35600069a0 2024-04-26T19:50:56.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7f0104f5c640 1 --2- 172.21.15.79:0/4239015491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f804ffd0 0x7f00f8037bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f00f8007ab0 tx=0x7f00fc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3985bfb8125884 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fd3874cc640 1 -- start start 2024-04-26T19:50:56.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:55.999+0000 7fcf9b59e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c071d00 con 0x7fcf9c1678e0 2024-04-26T19:50:56.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fcf99d9b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c1678e0 0x7fcf9c164da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fcf99d9b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c1678e0 0x7fcf9c164da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44928/0 (socket says 172.21.15.79:44928) 2024-04-26T19:50:56.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fcf99d9b640 1 -- 172.21.15.79:0/628435719 learned_addr learned my addr 172.21.15.79:0/628435719 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fcf9ad9d640 1 --2- 172.21.15.79:0/628435719 >> v2:172.21.15.83:3300/0 conn(0x7fcf9c14d060 0x7fcf9c0cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fcf99d9b640 1 -- 172.21.15.79:0/628435719 >> v2:172.21.15.83:3300/0 conn(0x7fcf9c14d060 msgr2=0x7fcf9c0cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fcf99d9b640 1 --2- 172.21.15.79:0/628435719 >> v2:172.21.15.83:3300/0 conn(0x7fcf9c14d060 0x7fcf9c0cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fcf99d9b640 1 -- 172.21.15.79:0/628435719 >> v1:172.21.15.83:6789/0 conn(0x7fcf9c14cc80 legacy=0x7fcf9c1671d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fcf99d9b640 1 -- 172.21.15.79:0/628435719 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c071fe0 con 0x7fcf9c1678e0 2024-04-26T19:50:56.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7fe25b159640 1 -- start start 2024-04-26T19:50:56.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fd3874cc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd38015dde0 0x7fd380167460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fd3874cc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0940 0x7fd3800b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fd3874cc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3800b0830 con 0x7fd3800d0dc0 2024-04-26T19:50:56.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fd3874cc640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd3800b0b10 con 0x7fd38015dde0 2024-04-26T19:50:56.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fd3874cc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd3800b0df0 con 0x7fd3800d0940 2024-04-26T19:50:56.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f356759e640 1 -- 172.21.15.79:0/1080092447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600069a0 msgr2=0x7f3560050540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f356759e640 1 --2- 172.21.15.79:0/1080092447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600069a0 0x7f3560050540 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f356806b410 tx=0x7f35680b7c80 comp rx=0 tx=0).stop 2024-04-26T19:50:56.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7f150dc62640 1 --2- 172.21.15.79:0/2601159887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500033ef0 0x7f150004fa90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f150806b410 tx=0x7f15080a6200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893ee713dc70601b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fe25b159640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe2540d61d0 0x7fe2540d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fe25b159640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254163220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f14fffff640 1 -- 172.21.15.79:0/2601159887 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080ae410 con 0x7f1500033ef0 2024-04-26T19:50:56.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f00f7fff640 1 -- 172.21.15.79:0/4239015491 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc037070 con 0x7f00f804ffd0 2024-04-26T19:50:56.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fcf99d9b640 1 --2- 172.21.15.79:0/628435719 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c1678e0 0x7fcf9c164da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009870 tx=0x7fcf8c02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc62aaff9222f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f356759e640 1 -- 172.21.15.79:0/1080092447 shutdown_connections 2024-04-26T19:50:56.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f356759e640 1 --2- 172.21.15.79:0/1080092447 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600069a0 0x7f3560050540 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f356759e640 1 -- 172.21.15.79:0/1080092447 >> 172.21.15.79:0/1080092447 conn(0x7f3560055350 msgr2=0x7f3560055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fd385a69640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fd3800d0dc0 0x7fd3801643c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:51116/0 (socket says 172.21.15.79:51116) 2024-04-26T19:50:56.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fd385a69640 1 -- 172.21.15.79:0/1737917331 learned_addr learned my addr 172.21.15.79:0/1737917331 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:56.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7f4b23fff640 1 --2- 172.21.15.79:0/63947449 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b24073ee0 0x7f4b2408ca60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4b140075f0 tx=0x7f4b140330e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58a26e9fbd913bc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7f4b237fe640 1 -- 172.21.15.79:0/63947449 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14033e30 con 0x7f4b24073ee0 2024-04-26T19:50:56.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7f4b237fe640 1 -- 172.21.15.79:0/63947449 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1400d0d0 con 0x7f4b24073ee0 2024-04-26T19:50:56.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.000+0000 7f4b237fe640 1 -- 172.21.15.79:0/63947449 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14033e30 con 0x7f4b24073ee0 2024-04-26T19:50:56.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f356759e640 1 -- 172.21.15.79:0/1080092447 shutdown_connections 2024-04-26T19:50:56.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f00f7fff640 1 -- 172.21.15.79:0/4239015491 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00fc032440 con 0x7f00f804ffd0 2024-04-26T19:50:56.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7fd384a67640 1 --2- 172.21.15.79:0/1737917331 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0940 0x7fd3800b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f00f7fff640 1 -- 172.21.15.79:0/4239015491 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc032810 con 0x7f00f804ffd0 2024-04-26T19:50:56.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f14fffff640 1 -- 172.21.15.79:0/2601159887 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15080aa070 con 0x7f1500033ef0 2024-04-26T19:50:56.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f14fffff640 1 -- 172.21.15.79:0/2601159887 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080ae830 con 0x7f1500033ef0 2024-04-26T19:50:56.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fd384a67640 1 -- 172.21.15.79:0/1737917331 >> v2:172.21.15.79:3300/0 conn(0x7fd38015dde0 msgr2=0x7fd380167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fd384a67640 1 --2- 172.21.15.79:0/1737917331 >> v2:172.21.15.79:3300/0 conn(0x7fd38015dde0 0x7fd380167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fd384a67640 1 -- 172.21.15.79:0/1737917331 >> v1:172.21.15.83:6789/0 conn(0x7fd3800d0dc0 legacy=0x7fd3801643c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fd384a67640 1 -- 172.21.15.79:0/1737917331 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3800b10d0 con 0x7fd3800d0940 2024-04-26T19:50:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f150e463640 1 -- 172.21.15.79:0/2601159887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500033ef0 msgr2=0x7f150004fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f150e463640 1 --2- 172.21.15.79:0/2601159887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500033ef0 0x7f150004fa90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f150806b410 tx=0x7f15080a6200 comp rx=0 tx=0).stop 2024-04-26T19:50:56.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f010675f640 1 -- 172.21.15.79:0/4239015491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f804ffd0 msgr2=0x7f00f8037bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f010675f640 1 --2- 172.21.15.79:0/4239015491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f804ffd0 0x7f00f8037bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f00f8007ab0 tx=0x7f00fc02a1c0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f010675f640 1 -- 172.21.15.79:0/4239015491 shutdown_connections 2024-04-26T19:50:56.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f010675f640 1 --2- 172.21.15.79:0/4239015491 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f804ffd0 0x7f00f8037bd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f010675f640 1 --2- 172.21.15.79:0/4239015491 >> v2:172.21.15.83:3300/0 conn(0x7f00f80ead00 0x7f00f804fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f010675f640 1 -- 172.21.15.79:0/4239015491 >> 172.21.15.79:0/4239015491 conn(0x7f00f8009fb0 msgr2=0x7f00f800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f4b2ab24640 1 -- 172.21.15.79:0/63947449 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b24073ee0 msgr2=0x7f4b2408ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7f4b2ab24640 1 --2- 172.21.15.79:0/63947449 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b24073ee0 0x7f4b2408ca60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4b140075f0 tx=0x7f4b140330e0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f150e463640 1 -- 172.21.15.79:0/2601159887 shutdown_connections 2024-04-26T19:50:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f150e463640 1 --2- 172.21.15.79:0/2601159887 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500033ef0 0x7f150004fa90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f150e463640 1 --2- 172.21.15.79:0/2601159887 >> v2:172.21.15.83:3300/0 conn(0x7f150000c0a0 0x7f1500038080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f150e463640 1 -- 172.21.15.79:0/2601159887 >> 172.21.15.79:0/2601159887 conn(0x7f1500009fb0 msgr2=0x7f150000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7fcf9959a640 1 -- 172.21.15.79:0/628435719 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c037070 con 0x7fcf9c1678e0 2024-04-26T19:50:56.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f010675f640 1 -- 172.21.15.79:0/4239015491 shutdown_connections 2024-04-26T19:50:56.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f010675f640 1 -- 172.21.15.79:0/4239015491 wait complete. 2024-04-26T19:50:56.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7fcf9959a640 1 -- 172.21.15.79:0/628435719 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf8c032440 con 0x7fcf9c1678e0 2024-04-26T19:50:56.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f150e463640 1 -- 172.21.15.79:0/2601159887 shutdown_connections 2024-04-26T19:50:56.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f150e463640 1 -- 172.21.15.79:0/2601159887 wait complete. 2024-04-26T19:50:56.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7fd384a67640 1 --2- 172.21.15.79:0/1737917331 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0940 0x7fd3800b3de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd3700097a0 tx=0x7fd37002a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b82907e6baea47de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.001+0000 7f356759e640 1 -- 172.21.15.79:0/1080092447 wait complete. 2024-04-26T19:50:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f356759e640 1 Processor -- start 2024-04-26T19:50:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7fcf9b59e640 1 -- 172.21.15.79:0/628435719 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c1678e0 msgr2=0x7fcf9c164da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7fcf9b59e640 1 --2- 172.21.15.79:0/628435719 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c1678e0 0x7fcf9c164da0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009870 tx=0x7fcf8c02a250 comp rx=0 tx=0).stop 2024-04-26T19:50:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fe25b159640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe25415f9f0 con 0x7fe254163760 2024-04-26T19:50:56.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fe25b159640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fe2540d5400 con 0x7fe2540d61d0 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fe25b159640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe2541605a0 con 0x7fe2540d4fa0 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fe253fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254163220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fe253fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254163220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44946/0 (socket says 172.21.15.79:44946) 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.002+0000 7fe253fff640 1 -- 172.21.15.79:0/3525116229 learned_addr learned my addr 172.21.15.79:0/3525116229 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7fe253fff640 1 -- 172.21.15.79:0/3525116229 >> v2:172.21.15.79:3300/0 conn(0x7fe2540d61d0 msgr2=0x7fe2540d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7fe253fff640 1 --2- 172.21.15.79:0/3525116229 >> v2:172.21.15.79:3300/0 conn(0x7fe2540d61d0 0x7fe2540d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7fe253fff640 1 -- 172.21.15.79:0/3525116229 >> v1:172.21.15.79:6789/0 conn(0x7fe254163760 legacy=0x7fe25406ef90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7fe253fff640 1 -- 172.21.15.79:0/3525116229 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe25415fc70 con 0x7fe2540d4fa0 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fe253fff640 1 --2- 172.21.15.79:0/3525116229 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254163220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe240009870 tx=0x7fe24002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2493c915329e87e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f4b2ab24640 1 -- 172.21.15.79:0/63947449 shutdown_connections 2024-04-26T19:50:56.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fcf9b59e640 1 -- 172.21.15.79:0/628435719 shutdown_connections 2024-04-26T19:50:56.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fd377fff640 1 -- 172.21.15.79:0/1737917331 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370037070 con 0x7fd3800d0940 2024-04-26T19:50:56.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fcf9b59e640 1 --2- 172.21.15.79:0/628435719 >> v2:172.21.15.83:3300/0 conn(0x7fcf9c14d060 0x7fcf9c0cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fd377fff640 1 -- 172.21.15.79:0/1737917331 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37002ae40 con 0x7fd3800d0940 2024-04-26T19:50:56.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fd377fff640 1 -- 172.21.15.79:0/1737917331 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370037070 con 0x7fd3800d0940 2024-04-26T19:50:56.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fcf9b59e640 1 --2- 172.21.15.79:0/628435719 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c1678e0 0x7fcf9c164da0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fcf9b59e640 1 -- 172.21.15.79:0/628435719 >> 172.21.15.79:0/628435719 conn(0x7fcf9c0b29e0 msgr2=0x7fcf9c0b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fe2537fe640 1 -- 172.21.15.79:0/3525116229 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240037070 con 0x7fe2540d4fa0 2024-04-26T19:50:56.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f010675f640 1 Processor -- start 2024-04-26T19:50:56.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f010675f640 1 -- start start 2024-04-26T19:50:56.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fe2537fe640 1 -- 172.21.15.79:0/3525116229 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe240032440 con 0x7fe2540d4fa0 2024-04-26T19:50:56.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fcf9b59e640 1 -- 172.21.15.79:0/628435719 shutdown_connections 2024-04-26T19:50:56.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fcf9b59e640 1 -- 172.21.15.79:0/628435719 wait complete. 2024-04-26T19:50:56.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7fe2537fe640 1 -- 172.21.15.79:0/3525116229 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240032810 con 0x7fe2540d4fa0 2024-04-26T19:50:56.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.003+0000 7f356759e640 1 -- start start 2024-04-26T19:50:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f150e463640 1 Processor -- start 2024-04-26T19:50:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f9de4f0a640 1 Processor -- start 2024-04-26T19:50:56.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f150e463640 1 -- start start 2024-04-26T19:50:56.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7fd3874cc640 1 -- 172.21.15.79:0/1737917331 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0940 msgr2=0x7fd3800b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7fd3874cc640 1 --2- 172.21.15.79:0/1737917331 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0940 0x7fd3800b3de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd3700097a0 tx=0x7fd37002a130 comp rx=0 tx=0).stop 2024-04-26T19:50:56.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f356759e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35600069a0 0x7f35600cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f356759e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35600345d0 0x7f35600dea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f356759e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600e75e0 0x7f35600e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f356759e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3560053bf0 con 0x7f35600069a0 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f356759e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3560034a20 con 0x7f35600e75e0 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f356759e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3560053d70 con 0x7f35600345d0 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f356659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35600069a0 0x7f35600cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f356659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35600069a0 0x7f35600cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38920/0 (socket says 172.21.15.79:38920) 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f356659c640 1 -- 172.21.15.79:0/2371247806 learned_addr learned my addr 172.21.15.79:0/2371247806 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f3565d9b640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35600345d0 0x7f35600dea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f3566d9d640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600e75e0 0x7f35600e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f010675f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f00f8038110 0x7f00f817ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7fd3874cc640 1 -- 172.21.15.79:0/1737917331 shutdown_connections 2024-04-26T19:50:56.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7fd3874cc640 1 --2- 172.21.15.79:0/1737917331 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0940 0x7fd3800b3de0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7fd3874cc640 1 --2- 172.21.15.79:0/1737917331 >> v2:172.21.15.79:3300/0 conn(0x7fd38015dde0 0x7fd380167460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7fd3874cc640 1 -- 172.21.15.79:0/1737917331 >> 172.21.15.79:0/1737917331 conn(0x7fd3800b39d0 msgr2=0x7fd38006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f3565d9b640 1 -- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600e75e0 msgr2=0x7f35600e34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f3565d9b640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600e75e0 0x7f35600e34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f3565d9b640 1 -- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35600069a0 msgr2=0x7f35600cb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f4b2ab24640 1 --2- 172.21.15.79:0/63947449 >> v2:172.21.15.79:3300/0 conn(0x7f4b2416c3a0 0x7f4b24071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f4b2ab24640 1 --2- 172.21.15.79:0/63947449 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b24073ee0 0x7f4b2408ca60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f4b2ab24640 1 -- 172.21.15.79:0/63947449 >> 172.21.15.79:0/63947449 conn(0x7f4b240b5060 msgr2=0x7f4b2406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f010675f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f804ffd0 0x7f00f8006e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7fe25b159640 1 -- 172.21.15.79:0/3525116229 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 msgr2=0x7fe254163220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7fe25b159640 1 --2- 172.21.15.79:0/3525116229 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254163220 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe240009870 tx=0x7fe24002a180 comp rx=0 tx=0).stop 2024-04-26T19:50:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fe25b159640 1 -- 172.21.15.79:0/3525116229 shutdown_connections 2024-04-26T19:50:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fe25b159640 1 --2- 172.21.15.79:0/3525116229 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254163220 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fe25b159640 1 --2- 172.21.15.79:0/3525116229 >> v2:172.21.15.79:3300/0 conn(0x7fe2540d61d0 0x7fe2540d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fe25b159640 1 -- 172.21.15.79:0/3525116229 >> 172.21.15.79:0/3525116229 conn(0x7fe2540d5630 msgr2=0x7fe254073820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f9de4f0a640 1 -- start start 2024-04-26T19:50:56.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f010575d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f00f8038110 0x7f00f817ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f010575d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f00f8038110 0x7f00f817ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44970/0 (socket says 172.21.15.79:44970) 2024-04-26T19:50:56.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fe25b159640 1 -- 172.21.15.79:0/3525116229 shutdown_connections 2024-04-26T19:50:56.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f010575d640 1 -- 172.21.15.79:0/1288438677 learned_addr learned my addr 172.21.15.79:0/1288438677 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f010675f640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f80ead00 0x7f00f8007370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fd3874cc640 1 -- 172.21.15.79:0/1737917331 shutdown_connections 2024-04-26T19:50:56.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fd3874cc640 1 -- 172.21.15.79:0/1737917331 wait complete. 2024-04-26T19:50:56.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f00f80547c0 con 0x7f00f804ffd0 2024-04-26T19:50:56.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f00f8038c90 con 0x7f00f80ead00 2024-04-26T19:50:56.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f00f8054940 con 0x7f00f8038110 2024-04-26T19:50:56.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fe25b159640 1 -- 172.21.15.79:0/3525116229 wait complete. 2024-04-26T19:50:56.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f150e463640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f150000c0a0 0x7f15000d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f150e463640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1500033ef0 0x7f15000caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f150e463640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500053c70 0x7f150004bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f150cc60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1500033ef0 0x7f15000caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f150cc60640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1500033ef0 0x7f15000caf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:48506/0 (socket says 172.21.15.79:48506) 2024-04-26T19:50:56.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f150d461640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f150000c0a0 0x7f15000d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f150d461640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f150000c0a0 0x7f15000d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:44982/0 (socket says 172.21.15.79:44982) 2024-04-26T19:50:56.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f9de4f0a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9de0161330 0x7f9de01649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f9de4f0a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9de006da80 0x7f9de0167eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f9de4f0a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de006df60 con 0x7f9de00b3a70 2024-04-26T19:50:56.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f9de4f0a640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f9de006e240 con 0x7f9de006da80 2024-04-26T19:50:56.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f9de4f0a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9de01641c0 con 0x7f9de0161330 2024-04-26T19:50:56.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f0104f5c640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f804ffd0 0x7f00f8006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f4b2ab24640 1 -- 172.21.15.79:0/63947449 shutdown_connections 2024-04-26T19:50:56.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.004+0000 7f4b2ab24640 1 -- 172.21.15.79:0/63947449 wait complete. 2024-04-26T19:50:56.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f9dded9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9de00b3a70 0x7f9de00b3e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:51128/0 (socket says 172.21.15.79:51128) 2024-04-26T19:50:56.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f9dded9d640 1 -- 172.21.15.79:0/752800920 learned_addr learned my addr 172.21.15.79:0/752800920 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:56.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f010575d640 1 -- 172.21.15.79:0/1288438677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f80ead00 msgr2=0x7f00f8007370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f3565d9b640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35600069a0 0x7f35600cb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f3565d9b640 1 -- 172.21.15.79:0/2371247806 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35600e5280 con 0x7f35600345d0 2024-04-26T19:50:56.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.005+0000 7f3566d9d640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600e75e0 0x7f35600e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:56.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f3565d9b640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35600345d0 0x7f35600dea00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f356807bbf0 tx=0x7f35680660f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f3556ffd640 1 -- 172.21.15.79:0/2371247806 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35680cc680 con 0x7f35600345d0 2024-04-26T19:50:56.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f3556ffd640 1 -- 172.21.15.79:0/2371247806 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35680cc820 con 0x7f35600345d0 2024-04-26T19:50:56.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7f3556ffd640 1 -- 172.21.15.79:0/2371247806 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35680ccbd0 con 0x7f35600345d0 2024-04-26T19:50:56.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f010575d640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f80ead00 0x7f00f8007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f010575d640 1 -- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f804ffd0 msgr2=0x7f00f8006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f010575d640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f804ffd0 0x7f00f8006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f010575d640 1 -- 172.21.15.79:0/1288438677 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00f817e840 con 0x7f00f8038110 2024-04-26T19:50:56.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f150dc62640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500053c70 0x7f150004bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f150dc62640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500053c70 0x7f150004bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38936/0 (socket says 172.21.15.79:38936) 2024-04-26T19:50:56.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f356659c640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35600069a0 0x7f35600cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:56.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f150dc62640 1 -- 172.21.15.79:0/2069743225 learned_addr learned my addr 172.21.15.79:0/2069743225 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1500037a90 con 0x7f1500053c70 2024-04-26T19:50:56.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1500037910 con 0x7f1500033ef0 2024-04-26T19:50:56.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f150004ffd0 con 0x7f150000c0a0 2024-04-26T19:50:56.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1262293224 0 0) 0x7f9de006df60 con 0x7f9de00b3a70 2024-04-26T19:50:56.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dc8003520 con 0x7f9de00b3a70 2024-04-26T19:50:56.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35600e5560 con 0x7f35600345d0 2024-04-26T19:50:56.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f35600e5b20 con 0x7f35600345d0 2024-04-26T19:50:56.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fd3874cc640 1 Processor -- start 2024-04-26T19:50:56.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fcf9b59e640 1 Processor -- start 2024-04-26T19:50:56.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.006+0000 7fcf9b59e640 1 -- start start 2024-04-26T19:50:56.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf9b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf9c14cc80 0x7fcf9c08bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf9b59e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c14d060 0x7fcf9c14ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf9b59e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c1678e0 0x7fcf9c14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf9b59e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c0d03c0 con 0x7fcf9c1678e0 2024-04-26T19:50:56.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf9b59e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c0d0240 con 0x7fcf9c14cc80 2024-04-26T19:50:56.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fd3874cc640 1 -- start start 2024-04-26T19:50:56.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf9b59e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c0d0540 con 0x7fcf9c14d060 2024-04-26T19:50:56.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf9a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf9c14cc80 0x7fcf9c08bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf9a59c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf9c14cc80 0x7fcf9c08bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:48528/0 (socket says 172.21.15.79:48528) 2024-04-26T19:50:56.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf9a59c640 1 -- 172.21.15.79:0/372351343 learned_addr learned my addr 172.21.15.79:0/372351343 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.050 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7fcf99d9b640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c14d060 0x7fcf9c14ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9dddd9b640 1 --2- 172.21.15.79:0/752800920 >> v2:172.21.15.79:3300/0 conn(0x7f9de006da80 0x7f9de0167eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 <== mon.1 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3145695835 0 0) 0x7f9dc8003520 con 0x7f9de00b3a70 2024-04-26T19:50:56.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 >> v2:172.21.15.83:3300/0 conn(0x7f9de0161330 msgr2=0x7f9de01649c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T19:50:56.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 --2- 172.21.15.79:0/752800920 >> v2:172.21.15.83:3300/0 conn(0x7f9de0161330 0x7f9de01649c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 >> v2:172.21.15.79:3300/0 conn(0x7f9de006da80 msgr2=0x7f9de0167eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 --2- 172.21.15.79:0/752800920 >> v2:172.21.15.79:3300/0 conn(0x7f9de006da80 0x7f9de0167eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de01644a0 con 0x7f9de00b3a70 2024-04-26T19:50:56.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 <== mon.1 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f9dd0001670 con 0x7f9de00b3a70 2024-04-26T19:50:56.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fcf9a59c640 1 -- 172.21.15.79:0/372351343 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c14d060 msgr2=0x7fcf9c14ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fcf9a59c640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c14d060 0x7fcf9c14ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fcf9a59c640 1 -- 172.21.15.79:0/372351343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c1678e0 msgr2=0x7fcf9c14bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fcf9a59c640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c1678e0 0x7fcf9c14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fcf9a59c640 1 -- 172.21.15.79:0/372351343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c08c350 con 0x7fcf9c14cc80 2024-04-26T19:50:56.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 >> v1:172.21.15.83:6789/0 conn(0x7f9de00b3a70 legacy=0x7f9de00b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fcf9a59c640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf9c14cc80 0x7fcf9c08bbe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c730 tx=0x7fcf9400cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fe25b159640 1 Processor -- start 2024-04-26T19:50:56.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80039b0 0x7f9dc8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc8004800 0x7f9dc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fe25b159640 1 -- start start 2024-04-26T19:50:56.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dc80051c0 0x7f9dc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9de006e240 con 0x7f9dc80039b0 2024-04-26T19:50:56.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.008+0000 7fcf8affd640 1 -- 172.21.15.79:0/372351343 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf94003b70 con 0x7fcf9c14cc80 2024-04-26T19:50:56.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9de01641c0 con 0x7f9dc8004800 2024-04-26T19:50:56.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9dc800c170 con 0x7f9dc80051c0 2024-04-26T19:50:56.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f9dded9d640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dc80051c0 0x7f9dc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f9dddd9b640 1 --2- 172.21.15.79:0/752800920 >> v2:172.21.15.79:3300/0 conn(0x7f9de006da80 0x7f9de0167eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:56.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fd3874cc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3800d0940 0x7fd3801510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fd3874cc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0dc0 0x7fd38015eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fd3874cc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38015dde0 0x7fd3801659b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f150dc62640 1 -- 172.21.15.79:0/2069743225 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f150000c0a0 msgr2=0x7f15000d47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f150dc62640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f150000c0a0 0x7f15000d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f150dc62640 1 -- 172.21.15.79:0/2069743225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1500033ef0 msgr2=0x7f15000caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f150dc62640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1500033ef0 0x7f15000caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f150dc62640 1 -- 172.21.15.79:0/2069743225 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15000d4d30 con 0x7f1500053c70 2024-04-26T19:50:56.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f150dc62640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500053c70 0x7f150004bb90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1508051930 tx=0x7f15080660f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f4b2ab24640 1 Processor -- start 2024-04-26T19:50:56.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.007+0000 7f4b2ab24640 1 -- start start 2024-04-26T19:50:56.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f150d461640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f150000c0a0 0x7f15000d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-26T19:50:56.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b2ab24640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b24073ee0 0x7f4b241644d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b2ab24640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b2416bfc0 0x7f4b2416d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b288c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b24073ee0 0x7f4b241644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b288c0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b24073ee0 0x7f4b241644d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:48542/0 (socket says 172.21.15.79:48542) 2024-04-26T19:50:56.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b2ab24640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b2416c3a0 0x7f4b2416df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b288c0640 1 -- 172.21.15.79:0/4257013245 learned_addr learned my addr 172.21.15.79:0/4257013245 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240cebc0 con 0x7f4b2416c3a0 2024-04-26T19:50:56.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4b240ce6a0 con 0x7f4b24073ee0 2024-04-26T19:50:56.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b23fff640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b2416bfc0 0x7f4b2416d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4b240ced40 con 0x7f4b2416bfc0 2024-04-26T19:50:56.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fd3874cc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd3800b05d0 con 0x7fd3800d0940 2024-04-26T19:50:56.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fd3874cc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3800b0450 con 0x7fd38015dde0 2024-04-26T19:50:56.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c14c490 con 0x7fcf9c14cc80 2024-04-26T19:50:56.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c084a50 con 0x7fcf9c14cc80 2024-04-26T19:50:56.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fcf8affd640 1 -- 172.21.15.79:0/372351343 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf94003d30 con 0x7fcf9c14cc80 2024-04-26T19:50:56.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fcf8affd640 1 -- 172.21.15.79:0/372351343 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9400aa90 con 0x7fcf9c14cc80 2024-04-26T19:50:56.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fd3874cc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd3801679a0 con 0x7fd3800d0dc0 2024-04-26T19:50:56.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.009+0000 7fd385a69640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38015dde0 0x7fd3801659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f4b290c1640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b2416c3a0 0x7f4b2416df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7fe25b159640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7fd384a67640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0dc0 0x7fd38015eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7fd384a67640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0dc0 0x7fd38015eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:45018/0 (socket says 172.21.15.79:45018) 2024-04-26T19:50:56.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7fd384a67640 1 -- 172.21.15.79:0/1480257842 learned_addr learned my addr 172.21.15.79:0/1480257842 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f4b288c0640 1 -- 172.21.15.79:0/4257013245 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b2416bfc0 msgr2=0x7f4b2416d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f010575d640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f00f8038110 0x7f00f817ad50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b3e0 tx=0x7f00f000b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f4b288c0640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b2416bfc0 0x7f4b2416d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f4b288c0640 1 -- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b2416c3a0 msgr2=0x7f4b2416df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f4b288c0640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b2416c3a0 0x7f4b2416df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f4b288c0640 1 -- 172.21.15.79:0/4257013245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2416c930 con 0x7f4b24073ee0 2024-04-26T19:50:56.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f4b23fff640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b2416bfc0 0x7f4b2416d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:56.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7fd385268640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3800d0940 0x7fd3801510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f9dddd9b640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc8004800 0x7f9dc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f9dded9d640 1 -- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc8004800 msgr2=0x7f9dc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f9dded9d640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dc8004800 0x7f9dc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f9dded9d640 1 -- 172.21.15.79:0/752800920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80039b0 msgr2=0x7f9dc8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:56.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f9dded9d640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80039b0 0x7f9dc8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f9dded9d640 1 -- 172.21.15.79:0/752800920 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc8003520 con 0x7f9dc80051c0 2024-04-26T19:50:56.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f14fdffb640 1 -- 172.21.15.79:0/2069743225 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080bd070 con 0x7f1500053c70 2024-04-26T19:50:56.064 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f14fdffb640 1 -- 172.21.15.79:0/2069743225 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15080addf0 con 0x7f1500053c70 2024-04-26T19:50:56.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15000d2660 con 0x7f1500053c70 2024-04-26T19:50:56.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f3556ffd640 1 -- 172.21.15.79:0/2371247806 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35680d4020 con 0x7f35600345d0 2024-04-26T19:50:56.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3560005710 con 0x7f35600345d0 2024-04-26T19:50:56.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f3556ffd640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f354806c9f0 0x7f354806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f4b288c0640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b24073ee0 0x7f4b241644d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00c530 tx=0x7f4b1c00c9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15000d2c20 con 0x7f1500053c70 2024-04-26T19:50:56.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f3556ffd640 1 -- 172.21.15.79:0/2371247806 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f356813a1b0 con 0x7f35600345d0 2024-04-26T19:50:56.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fd384a67640 1 -- 172.21.15.79:0/1480257842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38015dde0 msgr2=0x7fd3801659b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fd384a67640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38015dde0 0x7fd3801659b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fd384a67640 1 -- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3800d0940 msgr2=0x7fd3801510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fd384a67640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3800d0940 0x7fd3801510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fd384a67640 1 -- 172.21.15.79:0/1480257842 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3801f98f0 con 0x7fd3800d0dc0 2024-04-26T19:50:56.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fd385a69640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38015dde0 0x7fd3801659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:56.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f4b217fa640 1 -- 172.21.15.79:0/4257013245 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c017070 con 0x7f4b24073ee0 2024-04-26T19:50:56.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fe25b159640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2540d61d0 0x7fe254072de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fe25b159640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254163760 0x7fe254073320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fe25b159640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe254160230 con 0x7fe2540d61d0 2024-04-26T19:50:56.067 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fe25b159640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe254145ad0 con 0x7fe254163760 2024-04-26T19:50:56.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fe25b159640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe254163bd0 con 0x7fe2540d4fa0 2024-04-26T19:50:56.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f4b217fa640 1 -- 172.21.15.79:0/4257013245 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1c005ce0 con 0x7f4b24073ee0 2024-04-26T19:50:56.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2416cc10 con 0x7f4b24073ee0 2024-04-26T19:50:56.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f00f5ffb640 1 -- 172.21.15.79:0/1288438677 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0015020 con 0x7f00f8038110 2024-04-26T19:50:56.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f00f5ffb640 1 -- 172.21.15.79:0/1288438677 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f0011850 con 0x7f00f8038110 2024-04-26T19:50:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f4b217fa640 1 -- 172.21.15.79:0/4257013245 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c0072b0 con 0x7f4b24073ee0 2024-04-26T19:50:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.010+0000 7f00f5ffb640 1 -- 172.21.15.79:0/1288438677 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f0011c40 con 0x7f00f8038110 2024-04-26T19:50:56.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00f817eb20 con 0x7f00f8038110 2024-04-26T19:50:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b2416d1d0 con 0x7f4b24073ee0 2024-04-26T19:50:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fe253fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2540d61d0 0x7fe254072de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fe253fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2540d61d0 0x7fe254072de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38992/0 (socket says 172.21.15.79:38992) 2024-04-26T19:50:56.070 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7fe253fff640 1 -- 172.21.15.79:0/1910455409 learned_addr learned my addr 172.21.15.79:0/1910455409 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f00f817f0e0 con 0x7f00f8038110 2024-04-26T19:50:56.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.011+0000 7f9dded9d640 1 -- 172.21.15.79:0/752800920 reap_dead start 2024-04-26T19:50:56.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7f9dded9d640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dc80051c0 0x7f9dc800b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0005050 tx=0x7f9dc8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fe1357a157797ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fcf8affd640 1 -- 172.21.15.79:0/372351343 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcf9400ac30 con 0x7fcf9c14cc80 2024-04-26T19:50:56.071 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15000052c0 con 0x7f1500053c70 2024-04-26T19:50:56.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf80005290 con 0x7fcf9c14cc80 2024-04-26T19:50:56.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7f14fdffb640 1 -- 172.21.15.79:0/2069743225 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15080b9630 con 0x7f1500053c70 2024-04-26T19:50:56.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fd384a67640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0dc0 0x7fd38015eb90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd370000c00 tx=0x7fd3700046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fe258ef5640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fe2596f6640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254163760 0x7fe254073320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fe253fff640 1 -- 172.21.15.79:0/1910455409 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 msgr2=0x7fe254155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fcf8affd640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf7c06c9c0 0x7fcf7c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fe253fff640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fe253fff640 1 -- 172.21.15.79:0/1910455409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254163760 msgr2=0x7fe254073320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fe253fff640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254163760 0x7fe254073320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.013+0000 7f356659c640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f354806c9f0 0x7f354806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.013+0000 7fcf8affd640 1 -- 172.21.15.79:0/372351343 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcf9408bca0 con 0x7fcf9c14cc80 2024-04-26T19:50:56.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b240506c0 con 0x7f4b24073ee0 2024-04-26T19:50:56.075 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.012+0000 7fe253fff640 1 -- 172.21.15.79:0/1910455409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2541f5710 con 0x7fe2540d61d0 2024-04-26T19:50:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.013+0000 7fcf99d9b640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf7c06c9c0 0x7fcf7c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.013+0000 7f356659c640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f354806c9f0 0x7f354806eeb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3558001b90 tx=0x7f3558009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.013+0000 7fe253fff640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2540d61d0 0x7fe254072de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe240009870 tx=0x7fe2400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.076 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.013+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00f80052c0 con 0x7f00f8038110 2024-04-26T19:50:56.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.013+0000 7f4b217fa640 1 -- 172.21.15.79:0/4257013245 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b1c01a070 con 0x7f4b24073ee0 2024-04-26T19:50:56.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.013+0000 7fcf99d9b640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf7c06c9c0 0x7fcf7c06ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c034000 tx=0x7fcf8c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.077 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.014+0000 7f4b217fa640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b1006c8f0 0x7f4b1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.014+0000 7f14fdffb640 1 -- 172.21.15.79:0/2069743225 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f15080a6d10 con 0x7f1500053c70 2024-04-26T19:50:56.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.014+0000 7f14fdffb640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f14ec06c940 0x7f14ec06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.078 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.014+0000 7f4b23fff640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b1006c8f0 0x7f4b1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.014+0000 7f14fdffb640 1 -- 172.21.15.79:0/2069743225 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1508128ff0 con 0x7f1500053c70 2024-04-26T19:50:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.014+0000 7f150d461640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f14ec06c940 0x7f14ec06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.014+0000 7f4b217fa640 1 -- 172.21.15.79:0/4257013245 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4b1c08b1b0 con 0x7f4b24073ee0 2024-04-26T19:50:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.014+0000 7f4b23fff640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b1006c8f0 0x7f4b1006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b1403c000 tx=0x7f4b1400f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.015+0000 7f150d461640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f14ec06c940 0x7f14ec06ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f150004ecb0 tx=0x7f1504008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.016+0000 7f3556ffd640 1 -- 172.21.15.79:0/2371247806 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f35681081f0 con 0x7f35600345d0 2024-04-26T19:50:56.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.016+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0004c20 con 0x7f9dc80051c0 2024-04-26T19:50:56.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.016+0000 7fd375ffb640 1 -- 172.21.15.79:0/1480257842 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370043070 con 0x7fd3800d0dc0 2024-04-26T19:50:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.016+0000 7f9ddd59a640 1 -- 172.21.15.79:0/752800920 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd002f030 con 0x7f9dc80051c0 2024-04-26T19:50:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.016+0000 7fd375ffb640 1 -- 172.21.15.79:0/1480257842 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd37003c680 con 0x7fd3800d0dc0 2024-04-26T19:50:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.016+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3801f9bd0 con 0x7fd3800d0dc0 2024-04-26T19:50:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.016+0000 7fd375ffb640 1 -- 172.21.15.79:0/1480257842 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd370037070 con 0x7fd3800d0dc0 2024-04-26T19:50:56.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.016+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38015f5b0 con 0x7fd3800d0dc0 2024-04-26T19:50:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.017+0000 7f9de4f0a640 1 -- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dc80051c0 msgr2=0x7f9dc800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.017+0000 7f9de4f0a640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dc80051c0 0x7f9dc800b9c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0005050 tx=0x7f9dc8005830 comp rx=0 tx=0).stop 2024-04-26T19:50:56.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.017+0000 7f9de4f0a640 1 -- 172.21.15.79:0/752800920 shutdown_connections 2024-04-26T19:50:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.017+0000 7f9de4f0a640 1 --2- 172.21.15.79:0/752800920 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dc80051c0 0x7f9dc800b9c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.017+0000 7f9de4f0a640 1 -- 172.21.15.79:0/752800920 >> 172.21.15.79:0/752800920 conn(0x7f9de00b3660 msgr2=0x7f9de0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.017+0000 7f9de4f0a640 1 -- 172.21.15.79:0/752800920 shutdown_connections 2024-04-26T19:50:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.017+0000 7f9de4f0a640 1 -- 172.21.15.79:0/752800920 wait complete. 2024-04-26T19:50:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.018+0000 7fcf8affd640 1 -- 172.21.15.79:0/372351343 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcf94059e00 con 0x7fcf9c14cc80 2024-04-26T19:50:56.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.018+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3800506c0 con 0x7fd3800d0dc0 2024-04-26T19:50:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.018+0000 7f00f5ffb640 1 -- 172.21.15.79:0/1288438677 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00f0018020 con 0x7f00f8038110 2024-04-26T19:50:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.019+0000 7f14fdffb640 1 -- 172.21.15.79:0/2069743225 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f15080f6c00 con 0x7f1500053c70 2024-04-26T19:50:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.018+0000 7fe2517fa640 1 -- 172.21.15.79:0/1910455409 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240041070 con 0x7fe2540d61d0 2024-04-26T19:50:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.019+0000 7fe2517fa640 1 -- 172.21.15.79:0/1910455409 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe24003c870 con 0x7fe2540d61d0 2024-04-26T19:50:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.019+0000 7fe25b159640 1 -- 172.21.15.79:0/1910455409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2541f59f0 con 0x7fe2540d61d0 2024-04-26T19:50:56.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.019+0000 7fe25b159640 1 -- 172.21.15.79:0/1910455409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2541f5fb0 con 0x7fe2540d61d0 2024-04-26T19:50:56.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.019+0000 7fe2517fa640 1 -- 172.21.15.79:0/1910455409 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe240037040 con 0x7fe2540d61d0 2024-04-26T19:50:56.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.019+0000 7f00f5ffb640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f00e406c9c0 0x7f00e406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.019+0000 7f00f5ffb640 1 -- 172.21.15.79:0/1288438677 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f00f008c3b0 con 0x7f00f8038110 2024-04-26T19:50:56.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.019+0000 7f0104f5c640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f00e406c9c0 0x7f00e406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.020+0000 7f0104f5c640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f00e406c9c0 0x7f00e406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f00fc034000 tx=0x7f00fc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.020+0000 7f9de4f0a640 1 Processor -- start 2024-04-26T19:50:56.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.020+0000 7f9de4f0a640 1 -- start start 2024-04-26T19:50:56.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.020+0000 7fe23effd640 1 -- 172.21.15.79:0/1910455409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe22c005290 con 0x7fe2540d61d0 2024-04-26T19:50:56.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9de4f0a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80051c0 0x7f9de014fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f00f5ffb640 1 -- 172.21.15.79:0/1288438677 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f00f001f020 con 0x7f00f8038110 2024-04-26T19:50:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9de4f0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de0161a10 0x7f9de0161e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9de4f0a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9de00d1d10 0x7f9de00d2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9de4f0a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0161710 con 0x7f9dc80051c0 2024-04-26T19:50:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9dde59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80051c0 0x7f9de014fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9dde59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80051c0 0x7f9de014fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:38994/0 (socket says 172.21.15.79:38994) 2024-04-26T19:50:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9dde59c640 1 -- 172.21.15.79:0/272393110 learned_addr learned my addr 172.21.15.79:0/272393110 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9de4f0a640 1 -- 172.21.15.79:0/272393110 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9de0070f20 con 0x7f9de0161a10 2024-04-26T19:50:56.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9de4f0a640 1 -- 172.21.15.79:0/272393110 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9de0161890 con 0x7f9de00d1d10 2024-04-26T19:50:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9dded9d640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9de00d1d10 0x7f9de00d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.021+0000 7f9dddd9b640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de0161a10 0x7f9de0161e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.022+0000 7f9dded9d640 1 -- 172.21.15.79:0/272393110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de0161a10 msgr2=0x7f9de0161e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.022+0000 7f9dded9d640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de0161a10 0x7f9de0161e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.022+0000 7f9dded9d640 1 -- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80051c0 msgr2=0x7f9de014fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.022+0000 7f9dded9d640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80051c0 0x7f9de014fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.022+0000 7f9dded9d640 1 -- 172.21.15.79:0/272393110 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de00b2500 con 0x7f9de00d1d10 2024-04-26T19:50:56.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.022+0000 7fe2517fa640 1 -- 172.21.15.79:0/1910455409 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe240044070 con 0x7fe2540d61d0 2024-04-26T19:50:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.022+0000 7f9dde59c640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80051c0 0x7f9de014fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:56.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.022+0000 7f9dded9d640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9de00d1d10 0x7f9de00d2190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0004930 tx=0x7f9de0068bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.022+0000 7fe2517fa640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe23806c9f0 0x7fe23806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.023+0000 7fe2517fa640 1 -- 172.21.15.79:0/1910455409 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe2400ac7d0 con 0x7fe2540d61d0 2024-04-26T19:50:56.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.023+0000 7fe258ef5640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe23806c9f0 0x7fe23806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.023+0000 7fe258ef5640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe23806c9f0 0x7fe23806eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe248009990 tx=0x7fe248009110 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.023+0000 7f4b217fa640 1 -- 172.21.15.79:0/4257013245 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4b1c058d00 con 0x7f4b24073ee0 2024-04-26T19:50:56.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.025+0000 7fd375ffb640 1 -- 172.21.15.79:0/1480257842 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd370046070 con 0x7fd3800d0dc0 2024-04-26T19:50:56.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.026+0000 7f9dceffd640 1 -- 172.21.15.79:0/272393110 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0001d40 con 0x7f9de00d1d10 2024-04-26T19:50:56.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.026+0000 7fd375ffb640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd36406c8f0 0x7fd36406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.026+0000 7fd375ffb640 1 -- 172.21.15.79:0/1480257842 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd3700ac870 con 0x7fd3800d0dc0 2024-04-26T19:50:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.026+0000 7fd375ffb640 1 -- 172.21.15.79:0/1480257842 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd3700daca0 con 0x7fd3800d0dc0 2024-04-26T19:50:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.026+0000 7f9de4f0a640 1 -- 172.21.15.79:0/272393110 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de006e520 con 0x7f9de00d1d10 2024-04-26T19:50:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.026+0000 7f9dceffd640 1 -- 172.21.15.79:0/272393110 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd002f040 con 0x7f9de00d1d10 2024-04-26T19:50:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.026+0000 7f9dceffd640 1 -- 172.21.15.79:0/272393110 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0001d40 con 0x7f9de00d1d10 2024-04-26T19:50:56.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.026+0000 7f9de4f0a640 1 -- 172.21.15.79:0/272393110 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de006eae0 con 0x7f9de00d1d10 2024-04-26T19:50:56.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.028+0000 7f9dccff9640 1 -- 172.21.15.79:0/272393110 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9de006f5a0 con 0x7f9de00d1d10 2024-04-26T19:50:56.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.029+0000 7f9dceffd640 1 -- 172.21.15.79:0/272393110 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dd0005d10 con 0x7f9de00d1d10 2024-04-26T19:50:56.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.030+0000 7f9dceffd640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9dc006caf0 0x7f9dc006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:56.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.027+0000 7fd385268640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd36406c8f0 0x7fd36406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.028+0000 7fd385268640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd36406c8f0 0x7fd36406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b810 tx=0x7fd37c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.030+0000 7f9dde59c640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9dc006caf0 0x7f9dc006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:56.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.031+0000 7f9dceffd640 1 -- 172.21.15.79:0/272393110 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9dd0005ad0 con 0x7f9de00d1d10 2024-04-26T19:50:56.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.031+0000 7fe2517fa640 1 -- 172.21.15.79:0/1910455409 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe25415f9f0 con 0x7fe2540d61d0 2024-04-26T19:50:56.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.031+0000 7f9dde59c640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9dc006caf0 0x7f9dc006efb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8007e20 tx=0x7f9dd8007380 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:56.095 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.036+0000 7f9dceffd640 1 -- 172.21.15.79:0/272393110 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9dd0079100 con 0x7f9de00d1d10 2024-04-26T19:50:56.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.442+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3560054c10 con 0x7f35600345d0 2024-04-26T19:50:56.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.443+0000 7f3556ffd640 1 -- 172.21.15.79:0/2371247806 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3568107af0 con 0x7f35600345d0 2024-04-26T19:50:56.444 INFO:teuthology.orchestra.run.smithi079.stdout:55834574858 2024-04-26T19:50:56.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.444+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fcf80005580 con 0x7fcf9c14cc80 2024-04-26T19:50:56.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.444+0000 7fcf8affd640 1 -- 172.21.15.79:0/372351343 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcf94059700 con 0x7fcf9c14cc80 2024-04-26T19:50:56.445 INFO:teuthology.orchestra.run.smithi079.stdout:55834574857 2024-04-26T19:50:56.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f15000de9a0 con 0x7f1500053c70 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f354806c9f0 msgr2=0x7f354806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f354806c9f0 0x7f354806eeb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3558001b90 tx=0x7f3558009040 comp rx=0 tx=0).stop 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35600345d0 msgr2=0x7f35600dea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35600345d0 0x7f35600dea00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f356807bbf0 tx=0x7f35680660f0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 shutdown_connections 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f354806c9f0 0x7f354806eeb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35600e75e0 0x7f35600e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35600345d0 0x7f35600dea00 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f356807bbf0 tx=0x7f35680660f0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 --2- 172.21.15.79:0/2371247806 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35600069a0 0x7f35600cb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 >> 172.21.15.79:0/2371247806 conn(0x7f3560055350 msgr2=0x7f356004cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 shutdown_connections 2024-04-26T19:50:56.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f356759e640 1 -- 172.21.15.79:0/2371247806 wait complete. 2024-04-26T19:50:56.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.447+0000 7f14fdffb640 1 -- 172.21.15.79:0/2069743225 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f15080faf00 con 0x7f1500053c70 2024-04-26T19:50:56.449 INFO:teuthology.orchestra.run.smithi079.stdout:55834574859 2024-04-26T19:50:56.449 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4b240b4240 con 0x7f4b24073ee0 2024-04-26T19:50:56.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf7c06c9c0 msgr2=0x7fcf7c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf7c06c9c0 0x7fcf7c06ee80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c034000 tx=0x7fcf8c02e000 comp rx=0 tx=0).stop 2024-04-26T19:50:56.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf9c14cc80 msgr2=0x7fcf9c08bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf9c14cc80 0x7fcf9c08bbe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400c730 tx=0x7fcf9400cbf0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 shutdown_connections 2024-04-26T19:50:56.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcf7c06c9c0 0x7fcf7c06ee80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcf9c1678e0 0x7fcf9c14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcf9c14d060 0x7fcf9c14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 --2- 172.21.15.79:0/372351343 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcf9c14cc80 0x7fcf9c08bbe0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 >> 172.21.15.79:0/372351343 conn(0x7fcf9c0b29e0 msgr2=0x7fcf9c0b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.448+0000 7f4b217fa640 1 -- 172.21.15.79:0/4257013245 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4b1c05d000 con 0x7f4b24073ee0 2024-04-26T19:50:56.452 INFO:teuthology.orchestra.run.smithi079.stdout:55834574857 2024-04-26T19:50:56.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.450+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 shutdown_connections 2024-04-26T19:50:56.452 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.450+0000 7fcf9b59e640 1 -- 172.21.15.79:0/372351343 wait complete. 2024-04-26T19:50:56.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.450+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f14ec06c940 msgr2=0x7f14ec06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.450+0000 7f150e463640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f14ec06c940 0x7f14ec06ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f150004ecb0 tx=0x7f1504008040 comp rx=0 tx=0).stop 2024-04-26T19:50:56.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.450+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500053c70 msgr2=0x7f150004bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.450+0000 7f150e463640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500053c70 0x7f150004bb90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1508051930 tx=0x7f15080660f0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.453 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.450+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 shutdown_connections 2024-04-26T19:50:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f150e463640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f14ec06c940 0x7f14ec06ee00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f150e463640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1500053c70 0x7f150004bb90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f150e463640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1500033ef0 0x7f15000caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f150e463640 1 --2- 172.21.15.79:0/2069743225 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f150000c0a0 0x7f15000d47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 >> 172.21.15.79:0/2069743225 conn(0x7f1500009fb0 msgr2=0x7f15000ddde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 shutdown_connections 2024-04-26T19:50:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f150e463640 1 -- 172.21.15.79:0/2069743225 wait complete. 2024-04-26T19:50:56.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f00f80ea5b0 con 0x7f00f8038110 2024-04-26T19:50:56.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b1006c8f0 msgr2=0x7f4b1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f4b2ab24640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b1006c8f0 0x7f4b1006edb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4b1403c000 tx=0x7f4b1400f040 comp rx=0 tx=0).stop 2024-04-26T19:50:56.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.452+0000 7f00f5ffb640 1 -- 172.21.15.79:0/1288438677 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f00f005a480 con 0x7f00f8038110 2024-04-26T19:50:56.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b24073ee0 msgr2=0x7f4b241644d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.456 INFO:teuthology.orchestra.run.smithi079.stdout:55834574858 2024-04-26T19:50:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.451+0000 7f4b2ab24640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b24073ee0 0x7f4b241644d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00c530 tx=0x7f4b1c00c9f0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.452+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 shutdown_connections 2024-04-26T19:50:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.452+0000 7f4b2ab24640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b1006c8f0 0x7f4b1006edb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.452+0000 7f4b2ab24640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4b2416c3a0 0x7f4b2416df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.452+0000 7f4b2ab24640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4b2416bfc0 0x7f4b2416d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.452+0000 7f4b2ab24640 1 --2- 172.21.15.79:0/4257013245 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4b24073ee0 0x7f4b241644d0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00c530 tx=0x7f4b1c00c9f0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.452+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 >> 172.21.15.79:0/4257013245 conn(0x7f4b240b5060 msgr2=0x7f4b24169000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.452+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 shutdown_connections 2024-04-26T19:50:56.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.453+0000 7f4b2ab24640 1 -- 172.21.15.79:0/4257013245 wait complete. 2024-04-26T19:50:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.455+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f00e406c9c0 msgr2=0x7f00e406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.455+0000 7f010675f640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f00e406c9c0 0x7f00e406ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f00fc034000 tx=0x7f00fc02e000 comp rx=0 tx=0).stop 2024-04-26T19:50:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.455+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f00f8038110 msgr2=0x7f00f817ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.455+0000 7f010675f640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f00f8038110 0x7f00f817ad50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b3e0 tx=0x7f00f000b8a0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.458 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.456+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 shutdown_connections 2024-04-26T19:50:56.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.456+0000 7f010675f640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f00e406c9c0 0x7f00e406ee80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.456+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fd3800d0720 con 0x7fd3800d0dc0 2024-04-26T19:50:56.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.456+0000 7f010675f640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f00f80ead00 0x7f00f8007370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.456+0000 7f010675f640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f00f804ffd0 0x7f00f8006e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.456+0000 7f010675f640 1 --2- 172.21.15.79:0/1288438677 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f00f8038110 0x7f00f817ad50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.456+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 >> 172.21.15.79:0/1288438677 conn(0x7f00f8009fb0 msgr2=0x7f00f8033ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.456+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 shutdown_connections 2024-04-26T19:50:56.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.456+0000 7f010675f640 1 -- 172.21.15.79:0/1288438677 wait complete. 2024-04-26T19:50:56.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.457+0000 7fd375ffb640 1 -- 172.21.15.79:0/1480257842 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd37007a940 con 0x7fd3800d0dc0 2024-04-26T19:50:56.460 INFO:teuthology.orchestra.run.smithi079.stdout:55834574859 2024-04-26T19:50:56.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.459+0000 7f9dccff9640 1 -- 172.21.15.79:0/272393110 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9de01623d0 con 0x7f9de00d1d10 2024-04-26T19:50:56.461 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.460+0000 7f9dceffd640 1 -- 172.21.15.79:0/272393110 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9dd007d400 con 0x7f9de00d1d10 2024-04-26T19:50:56.461 INFO:teuthology.orchestra.run.smithi079.stdout:55834574857 2024-04-26T19:50:56.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.460+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd36406c8f0 msgr2=0x7fd36406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fd3874cc640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd36406c8f0 0x7fd36406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b810 tx=0x7fd37c008040 comp rx=0 tx=0).stop 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0dc0 msgr2=0x7fd38015eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fd3874cc640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0dc0 0x7fd38015eb90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd370000c00 tx=0x7fd3700046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 shutdown_connections 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fd3874cc640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd36406c8f0 0x7fd36406edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fd3874cc640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd38015dde0 0x7fd3801659b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fd3874cc640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3800d0dc0 0x7fd38015eb90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fd3874cc640 1 --2- 172.21.15.79:0/1480257842 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3800d0940 0x7fd3801510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 >> 172.21.15.79:0/1480257842 conn(0x7fd3800b39d0 msgr2=0x7fd3800b3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.463 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.461+0000 7fe23effd640 1 -- 172.21.15.79:0/1910455409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fe22c005580 con 0x7fe2540d61d0 2024-04-26T19:50:56.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.462+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 shutdown_connections 2024-04-26T19:50:56.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.462+0000 7fd3874cc640 1 -- 172.21.15.79:0/1480257842 wait complete. 2024-04-26T19:50:56.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.462+0000 7fe2517fa640 1 -- 172.21.15.79:0/1910455409 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe24007a260 con 0x7fe2540d61d0 2024-04-26T19:50:56.464 INFO:teuthology.orchestra.run.smithi079.stdout:55834574856 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.464+0000 7f9dccff9640 1 -- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9dc006caf0 msgr2=0x7f9dc006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.464+0000 7f9dccff9640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9dc006caf0 0x7f9dc006efb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8007e20 tx=0x7f9dd8007380 comp rx=0 tx=0).stop 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.464+0000 7f9dccff9640 1 -- 172.21.15.79:0/272393110 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9de00d1d10 msgr2=0x7f9de00d2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.464+0000 7f9dccff9640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9de00d1d10 0x7f9de00d2190 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0004930 tx=0x7f9de0068bf0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.465+0000 7f9dccff9640 1 -- 172.21.15.79:0/272393110 shutdown_connections 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.465+0000 7f9dccff9640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9dc006caf0 0x7f9dc006efb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.465+0000 7f9dccff9640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9de00d1d10 0x7f9de00d2190 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.465+0000 7f9dccff9640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9de0161a10 0x7f9de0161e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.465+0000 7f9dccff9640 1 --2- 172.21.15.79:0/272393110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dc80051c0 0x7f9de014fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.465+0000 7f9dccff9640 1 -- 172.21.15.79:0/272393110 >> 172.21.15.79:0/272393110 conn(0x7f9de00b3660 msgr2=0x7f9de006c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.465+0000 7f9dccff9640 1 -- 172.21.15.79:0/272393110 shutdown_connections 2024-04-26T19:50:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.465+0000 7f9dccff9640 1 -- 172.21.15.79:0/272393110 wait complete. 2024-04-26T19:50:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 -- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe23806c9f0 msgr2=0x7fe23806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe23806c9f0 0x7fe23806eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe248009990 tx=0x7fe248009110 comp rx=0 tx=0).stop 2024-04-26T19:50:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 -- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2540d61d0 msgr2=0x7fe254072de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:56.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2540d61d0 0x7fe254072de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe240009870 tx=0x7fe2400046c0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 -- 172.21.15.79:0/1910455409 shutdown_connections 2024-04-26T19:50:56.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe23806c9f0 0x7fe23806eeb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe254163760 0x7fe254073320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe2540d61d0 0x7fe254072de0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 --2- 172.21.15.79:0/1910455409 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe2540d4fa0 0x7fe254155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:56.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.466+0000 7fe23effd640 1 -- 172.21.15.79:0/1910455409 >> 172.21.15.79:0/1910455409 conn(0x7fe2540d5630 msgr2=0x7fe254142e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:56.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.467+0000 7fe23effd640 1 -- 172.21.15.79:0/1910455409 shutdown_connections 2024-04-26T19:50:56.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:56.467+0000 7fe23effd640 1 -- 172.21.15.79:0/1910455409 wait complete. 2024-04-26T19:50:56.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-26T19:50:56.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.5 2024-04-26T19:50:56.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.7 2024-04-26T19:50:56.478 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.1 2024-04-26T19:50:56.481 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.2 2024-04-26T19:50:56.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574859 for osd.0 2024-04-26T19:50:56.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.6 2024-04-26T19:50:56.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.4 2024-04-26T19:50:57.470 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-26T19:50:57.474 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-26T19:50:57.476 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-26T19:50:57.478 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T19:50:57.482 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T19:50:57.483 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T19:50:57.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-26T19:50:57.486 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-26T19:50:57.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.628+0000 7f8ef6425640 1 Processor -- start 2024-04-26T19:50:57.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.628+0000 7f8ef6425640 1 -- start start 2024-04-26T19:50:57.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.629+0000 7f8ef6425640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8ef00ce6d0 0x7f8ef00d1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.629+0000 7f8ef6425640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8ef0140b90 0x7f8ef0141000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.630+0000 7f8ef6425640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef0141900 con 0x7f8ef013e2a0 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.630+0000 7f8ef6425640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef00d0c80 con 0x7f8ef0140b90 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.630+0000 7f8ef6425640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef00d0630 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.630+0000 7f8ef49c2640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8ef0140b90 0x7f8ef0141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.630+0000 7f8ef49c2640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f8ef0140b90 0x7f8ef0141000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46104/0 (socket says 172.21.15.79:46104) 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.630+0000 7f8ef49c2640 1 -- 172.21.15.79:0/3723923789 learned_addr learned my addr 172.21.15.79:0/3723923789 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.630+0000 7f8ef49c2640 1 -- 172.21.15.79:0/3723923789 >> v2:172.21.15.83:3300/0 conn(0x7f8ef00ce6d0 msgr2=0x7f8ef00d1110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.630+0000 7f8ef49c2640 1 --2- 172.21.15.79:0/3723923789 >> v2:172.21.15.83:3300/0 conn(0x7f8ef00ce6d0 0x7f8ef00d1110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.630+0000 7f8ef49c2640 1 -- 172.21.15.79:0/3723923789 >> v1:172.21.15.83:6789/0 conn(0x7f8ef013e2a0 legacy=0x7f8ef00cdfc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.631+0000 7f8ef49c2640 1 -- 172.21.15.79:0/3723923789 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef00d0910 con 0x7f8ef0140b90 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.632+0000 7f8ef49c2640 1 --2- 172.21.15.79:0/3723923789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef0140b90 0x7f8ef0141000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0145170 tx=0x7f8ee804c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268d7c568b86df59 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.634+0000 7f8eeeffd640 1 -- 172.21.15.79:0/3723923789 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8053400 con 0x7f8ef0140b90 2024-04-26T19:50:57.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.634+0000 7f8eeeffd640 1 -- 172.21.15.79:0/3723923789 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee80535a0 con 0x7f8ef0140b90 2024-04-26T19:50:57.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.639+0000 7f8ef6425640 1 -- 172.21.15.79:0/3723923789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef0140b90 msgr2=0x7f8ef0141000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.640+0000 7f8ef6425640 1 --2- 172.21.15.79:0/3723923789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef0140b90 0x7f8ef0141000 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0145170 tx=0x7f8ee804c8f0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.641+0000 7f8ef6425640 1 -- 172.21.15.79:0/3723923789 shutdown_connections 2024-04-26T19:50:57.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.641+0000 7f8ef6425640 1 --2- 172.21.15.79:0/3723923789 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef0140b90 0x7f8ef0141000 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.641+0000 7f8ef6425640 1 --2- 172.21.15.79:0/3723923789 >> v2:172.21.15.83:3300/0 conn(0x7f8ef00ce6d0 0x7f8ef00d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.641+0000 7f8ef6425640 1 -- 172.21.15.79:0/3723923789 >> 172.21.15.79:0/3723923789 conn(0x7f8ef00b2e00 msgr2=0x7f8ef00b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:57.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.641+0000 7f8ef6425640 1 -- 172.21.15.79:0/3723923789 shutdown_connections 2024-04-26T19:50:57.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.641+0000 7f8ef6425640 1 -- 172.21.15.79:0/3723923789 wait complete. 2024-04-26T19:50:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.643+0000 7f8ef6425640 1 Processor -- start 2024-04-26T19:50:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.643+0000 7f8ef6425640 1 -- start start 2024-04-26T19:50:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.644+0000 7f8ef6425640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef00ce6d0 0x7f8ef0150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.644+0000 7f8ef6425640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ef013e2a0 0x7f8ef0150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.644+0000 7f8ef6425640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ef0140b90 0x7f8ef013e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.644+0000 7f8ef6425640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef01416c0 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.644+0000 7f8ef6425640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0141540 con 0x7f8ef013e2a0 2024-04-26T19:50:57.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.644+0000 7f8ef6425640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef013e680 con 0x7f8ef0140b90 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.645+0000 7f8eef7fe640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ef013e2a0 0x7f8ef0150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.645+0000 7f8ef49c2640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ef0140b90 0x7f8ef013e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.645+0000 7f8ef49c2640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ef0140b90 0x7f8ef013e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:46542/0 (socket says 172.21.15.79:46542) 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.645+0000 7f8ef49c2640 1 -- 172.21.15.79:0/3948405157 learned_addr learned my addr 172.21.15.79:0/3948405157 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.645+0000 7f8eeffff640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef00ce6d0 0x7f8ef0150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.646+0000 7f8eeffff640 1 -- 172.21.15.79:0/3948405157 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ef0140b90 msgr2=0x7f8ef013e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.646+0000 7f8eeffff640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ef0140b90 0x7f8ef013e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.646+0000 7f8eeffff640 1 -- 172.21.15.79:0/3948405157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ef013e2a0 msgr2=0x7f8ef0150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.646+0000 7f8eeffff640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ef013e2a0 0x7f8ef0150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.646+0000 7f8eeffff640 1 -- 172.21.15.79:0/3948405157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef0140480 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.646+0000 7f8eeffff640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef00ce6d0 0x7f8ef0150bd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400b750 tx=0x7f8ee400bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.648+0000 7f8eecff9640 1 -- 172.21.15.79:0/3948405157 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4019070 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.648+0000 7f8ef6425640 1 -- 172.21.15.79:0/3948405157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef0140760 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.648+0000 7f8ef6425640 1 -- 172.21.15.79:0/3948405157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef014fb10 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.650+0000 7f8eecff9640 1 -- 172.21.15.79:0/3948405157 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee400c720 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.651+0000 7f8eecff9640 1 -- 172.21.15.79:0/3948405157 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400cb70 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.652+0000 7f8ef6425640 1 -- 172.21.15.79:0/3948405157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ecc005290 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.659+0000 7f8eecff9640 1 -- 172.21.15.79:0/3948405157 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8ee401c070 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.659+0000 7f8eecff9640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8ed406c9c0 0x7f8ed406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.659+0000 7f8eecff9640 1 -- 172.21.15.79:0/3948405157 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8ee408b320 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.662+0000 7f8eef7fe640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8ed406c9c0 0x7f8ed406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.663+0000 7f8eef7fe640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8ed406c9c0 0x7f8ed406ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0150b80 tx=0x7f8ee0009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.663+0000 7f8eecff9640 1 -- 172.21.15.79:0/3948405157 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8ee4058e70 con 0x7f8ef00ce6d0 2024-04-26T19:50:57.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.680+0000 7f8726339640 1 Processor -- start 2024-04-26T19:50:57.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.680+0000 7f8726339640 1 -- start start 2024-04-26T19:50:57.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.682+0000 7f8726339640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8720071f20 0x7f87200738a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.682+0000 7f8726339640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f872016f610 0x7f87200cf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.682+0000 7f8726339640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87200cda50 con 0x7f872016f230 2024-04-26T19:50:57.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.682+0000 7f8726339640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f8720072690 con 0x7f872016f610 2024-04-26T19:50:57.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.682+0000 7f8726339640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f87200cd490 con 0x7f8720071f20 2024-04-26T19:50:57.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.682+0000 7f87248d6640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f872016f610 0x7f87200cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.682+0000 7f87248d6640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f872016f610 0x7f87200cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46136/0 (socket says 172.21.15.79:46136) 2024-04-26T19:50:57.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.682+0000 7f87248d6640 1 -- 172.21.15.79:0/2281429194 learned_addr learned my addr 172.21.15.79:0/2281429194 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.683+0000 7f871f7fe640 1 --2- 172.21.15.79:0/2281429194 >> v2:172.21.15.83:3300/0 conn(0x7f8720071f20 0x7f87200738a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.684 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.683+0000 7f87248d6640 1 -- 172.21.15.79:0/2281429194 >> v2:172.21.15.83:3300/0 conn(0x7f8720071f20 msgr2=0x7f87200738a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.683+0000 7f87248d6640 1 --2- 172.21.15.79:0/2281429194 >> v2:172.21.15.83:3300/0 conn(0x7f8720071f20 0x7f87200738a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.683+0000 7f87248d6640 1 -- 172.21.15.79:0/2281429194 >> v1:172.21.15.83:6789/0 conn(0x7f872016f230 legacy=0x7f8720071810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T19:50:57.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.683+0000 7f87248d6640 1 -- 172.21.15.79:0/2281429194 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87200cd770 con 0x7f872016f610 2024-04-26T19:50:57.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.684+0000 7f1299893640 1 Processor -- start 2024-04-26T19:50:57.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.684+0000 7f1299893640 1 -- start start 2024-04-26T19:50:57.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.684+0000 7f87248d6640 1 --2- 172.21.15.79:0/2281429194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 0x7f87200cf420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87200b5440 tx=0x7f871802bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857aafcb5d6e0623 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.684+0000 7f871effd640 1 -- 172.21.15.79:0/2281429194 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8718039070 con 0x7f872016f610 2024-04-26T19:50:57.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.685+0000 7f871effd640 1 -- 172.21.15.79:0/2281429194 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f871802c810 con 0x7f872016f610 2024-04-26T19:50:57.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f1299893640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940b1be0 0x7f129408ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f8726339640 1 -- 172.21.15.79:0/2281429194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 msgr2=0x7f87200cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7efdb7fff640 1 Processor -- start 2024-04-26T19:50:57.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f8726339640 1 --2- 172.21.15.79:0/2281429194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 0x7f87200cf420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f87200b5440 tx=0x7f871802bd10 comp rx=0 tx=0).stop 2024-04-26T19:50:57.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f8726339640 1 -- 172.21.15.79:0/2281429194 shutdown_connections 2024-04-26T19:50:57.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.684+0000 7fc7ca97f640 1 Processor -- start 2024-04-26T19:50:57.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.684+0000 7fc7ca97f640 1 -- start start 2024-04-26T19:50:57.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.685+0000 7fc7ca97f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc7bc006a30 0x7fc7bc006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.685+0000 7fc7ca97f640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7bc053720 con 0x7fc7bc0ead00 2024-04-26T19:50:57.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.685+0000 7fc7ca97f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7bc053a00 con 0x7fc7bc005ee0 2024-04-26T19:50:57.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.685+0000 7fc7ca97f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc7bc054330 con 0x7fc7bc006a30 2024-04-26T19:50:57.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c3fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc7bc0ead00 0x7fc7bc04fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53058/0 (socket says 172.21.15.79:53058) 2024-04-26T19:50:57.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c3fff640 1 -- 172.21.15.79:0/4269013477 learned_addr learned my addr 172.21.15.79:0/4269013477 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:57.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c8f1c640 1 --2- 172.21.15.79:0/4269013477 >> v2:172.21.15.83:3300/0 conn(0x7fc7bc006a30 0x7fc7bc006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 661178211 0 0) 0x7fc7bc053a00 con 0x7fc7bc005ee0 2024-04-26T19:50:57.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7b4003520 con 0x7fc7bc005ee0 2024-04-26T19:50:57.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f8726339640 1 --2- 172.21.15.79:0/2281429194 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 0x7f87200cf420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f8726339640 1 --2- 172.21.15.79:0/2281429194 >> v2:172.21.15.83:3300/0 conn(0x7f8720071f20 0x7f87200738a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f8726339640 1 -- 172.21.15.79:0/2281429194 >> 172.21.15.79:0/2281429194 conn(0x7f87200b4e80 msgr2=0x7f872006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:57.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f8726339640 1 -- 172.21.15.79:0/2281429194 shutdown_connections 2024-04-26T19:50:57.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 <== mon.1 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2276969240 0 0) 0x7fc7b4003520 con 0x7fc7bc005ee0 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 >> v2:172.21.15.83:3300/0 conn(0x7fc7bc006a30 msgr2=0x7fc7bc006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c2ffd640 1 --2- 172.21.15.79:0/4269013477 >> v2:172.21.15.83:3300/0 conn(0x7fc7bc006a30 0x7fc7bc006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 >> v1:172.21.15.79:6789/0 conn(0x7fc7bc0ead00 legacy=0x7fc7bc04fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 --> v1:172.21.15.83:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc053ce0 con 0x7fc7bc005ee0 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 <== mon.1 v1:172.21.15.83:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fc7c4050f10 con 0x7fc7bc005ee0 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 >> v1:172.21.15.83:6789/0 conn(0x7fc7bc005ee0 legacy=0x7fc7bc006320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c2ffd640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b40039b0 0x7fc7b4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c2ffd640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b4004800 0x7fc7b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c3fff640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b40039b0 0x7fc7b4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c2ffd640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc7b40051c0 0x7fc7b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c37fe640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b4004800 0x7fc7b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc054330 con 0x7fc7b40039b0 2024-04-26T19:50:57.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b400bfe0 con 0x7fc7b4004800 2024-04-26T19:50:57.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc7b400c1e0 con 0x7fc7b40051c0 2024-04-26T19:50:57.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f1299893640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1294073ee0 0x7f129416c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f1299893640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12940b23d0 con 0x7f12940cdad0 2024-04-26T19:50:57.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f1299893640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f12940cd8d0 con 0x7f1294073ee0 2024-04-26T19:50:57.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f1299893640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f12940b26b0 con 0x7f12940b1be0 2024-04-26T19:50:57.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7fc7c8f1c640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc7b40051c0 0x7fc7b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f12927fc640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f12940cdad0 0x7f12940737d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53072/0 (socket says 172.21.15.79:53072) 2024-04-26T19:50:57.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f12927fc640 1 -- 172.21.15.79:0/2578607459 learned_addr learned my addr 172.21.15.79:0/2578607459 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:57.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f1292ffd640 1 --2- 172.21.15.79:0/2578607459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940b1be0 0x7f129408ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7f12937fe640 1 --2- 172.21.15.79:0/2578607459 >> v2:172.21.15.83:3300/0 conn(0x7f1294073ee0 0x7f129416c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7f1292ffd640 1 -- 172.21.15.79:0/2578607459 >> v2:172.21.15.83:3300/0 conn(0x7f1294073ee0 msgr2=0x7f129416c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7f1292ffd640 1 --2- 172.21.15.79:0/2578607459 >> v2:172.21.15.83:3300/0 conn(0x7f1294073ee0 0x7f129416c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7f1292ffd640 1 -- 172.21.15.79:0/2578607459 >> v1:172.21.15.79:6789/0 conn(0x7f12940cdad0 legacy=0x7f12940737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7f1292ffd640 1 -- 172.21.15.79:0/2578607459 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12940ce500 con 0x7f12940b1be0 2024-04-26T19:50:57.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7f95a9d35640 1 Processor -- start 2024-04-26T19:50:57.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7f70a1cfc640 1 Processor -- start 2024-04-26T19:50:57.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7f70a1cfc640 1 -- start start 2024-04-26T19:50:57.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7f8726339640 1 -- 172.21.15.79:0/2281429194 wait complete. 2024-04-26T19:50:57.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7fce45d04640 1 Processor -- start 2024-04-26T19:50:57.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7fce45d04640 1 -- start start 2024-04-26T19:50:57.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7fc7c37fe640 1 -- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc7b40051c0 msgr2=0x7fc7b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7fc7c37fe640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc7b40051c0 0x7fc7b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7fc7c37fe640 1 -- 172.21.15.79:0/4269013477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b40039b0 msgr2=0x7fc7b4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7fc7c37fe640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b40039b0 0x7fc7b4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7fc7c37fe640 1 -- 172.21.15.79:0/4269013477 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b4003520 con 0x7fc7b4004800 2024-04-26T19:50:57.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7f95a9d35640 1 -- start start 2024-04-26T19:50:57.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f95a9d35640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 0x7f95a4073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f95a9d35640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f95a40d1930 0x7f95a40b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f95a9d35640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a4071bf0 con 0x7f95a4073fe0 2024-04-26T19:50:57.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f95a9d35640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f95a4071ed0 con 0x7f95a40d1930 2024-04-26T19:50:57.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f95a9d35640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f95a40721b0 con 0x7f95a415b1e0 2024-04-26T19:50:57.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f95a9534640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f95a40d1930 0x7f95a40b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f95a9534640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f95a40d1930 0x7f95a40b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46162/0 (socket says 172.21.15.79:46162) 2024-04-26T19:50:57.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f95a9534640 1 -- 172.21.15.79:0/2157411432 learned_addr learned my addr 172.21.15.79:0/2157411432 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f8726339640 1 Processor -- start 2024-04-26T19:50:57.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f70a1cfc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f709c0d1500 0x7f709c14cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f70a1cfc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f709c0736f0 0x7f709c14d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f70a1cfc640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f709c073d10 con 0x7f709c0d18e0 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f70a1cfc640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f709c0b1d80 con 0x7f709c0736f0 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f70a1cfc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f709c073ff0 con 0x7f709c0d1500 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f709bfff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f709c0736f0 0x7f709c14d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f709bfff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f709c0736f0 0x7f709c14d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46154/0 (socket says 172.21.15.79:46154) 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f709bfff640 1 -- 172.21.15.79:0/4097382834 learned_addr learned my addr 172.21.15.79:0/4097382834 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.686+0000 7efdb7fff640 1 -- start start 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7efdb7fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efdb815b1e0 0x7efdb8073aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7efdb7fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efdb8073fe0 0x7efdb80d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7efdb7fff640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdb80d1bb0 con 0x7efdb80d1760 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7efdb7fff640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7efdb8071bf0 con 0x7efdb815b1e0 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.687+0000 7efdb7fff640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efdb8071ed0 con 0x7efdb8073fe0 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7efdb67fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efdb8073fe0 0x7efdb80d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7efdb67fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efdb8073fe0 0x7efdb80d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:36238/0 (socket says 172.21.15.79:36238) 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7efdb67fc640 1 -- 172.21.15.79:0/3615469766 learned_addr learned my addr 172.21.15.79:0/3615469766 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7efdb6ffd640 1 --2- 172.21.15.79:0/3615469766 >> v2:172.21.15.79:3300/0 conn(0x7efdb815b1e0 0x7efdb8073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7efdb67fc640 1 -- 172.21.15.79:0/3615469766 >> v2:172.21.15.79:3300/0 conn(0x7efdb815b1e0 msgr2=0x7efdb8073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7efdb67fc640 1 --2- 172.21.15.79:0/3615469766 >> v2:172.21.15.79:3300/0 conn(0x7efdb815b1e0 0x7efdb8073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7efdb67fc640 1 -- 172.21.15.79:0/3615469766 >> v1:172.21.15.83:6789/0 conn(0x7efdb80d1760 legacy=0x7efdb815ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7efdb67fc640 1 -- 172.21.15.79:0/3615469766 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb80721b0 con 0x7efdb8073fe0 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7efdb6ffd640 1 --2- 172.21.15.79:0/3615469766 >> v2:172.21.15.79:3300/0 conn(0x7efdb815b1e0 0x7efdb8073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:57.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7efdb67fc640 1 --2- 172.21.15.79:0/3615469766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb8073fe0 0x7efdb80d1220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efdb8073310 tx=0x7efdb002a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b242b7352b0bd85c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7fc7c8f1c640 1 -- 172.21.15.79:0/4269013477 reap_dead start 2024-04-26T19:50:57.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.688+0000 7f1292ffd640 1 --2- 172.21.15.79:0/2578607459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940b1be0 0x7f129408ca60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12840097a0 tx=0x7f128402a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953aa992a7bd6076 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f1291ffb640 1 -- 172.21.15.79:0/2578607459 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1284037070 con 0x7f12940b1be0 2024-04-26T19:50:57.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f1291ffb640 1 -- 172.21.15.79:0/2578607459 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1284032440 con 0x7f12940b1be0 2024-04-26T19:50:57.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f1291ffb640 1 -- 172.21.15.79:0/2578607459 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12840327f0 con 0x7f12940b1be0 2024-04-26T19:50:57.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f709bfff640 1 -- 172.21.15.79:0/4097382834 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f709c0d1500 msgr2=0x7f709c14cad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T19:50:57.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f709bfff640 1 --2- 172.21.15.79:0/4097382834 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f709c0d1500 0x7f709c14cad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.689+0000 7f709bfff640 1 -- 172.21.15.79:0/4097382834 >> v1:172.21.15.79:6789/0 conn(0x7f709c0d18e0 legacy=0x7f709c072fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f709bfff640 1 -- 172.21.15.79:0/4097382834 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c0b0570 con 0x7f709c0736f0 2024-04-26T19:50:57.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7efdb5ffb640 1 -- 172.21.15.79:0/3615469766 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0036070 con 0x7efdb8073fe0 2024-04-26T19:50:57.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7efdb5ffb640 1 -- 172.21.15.79:0/3615469766 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb0031600 con 0x7efdb8073fe0 2024-04-26T19:50:57.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7efdb5ffb640 1 -- 172.21.15.79:0/3615469766 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0031a10 con 0x7efdb8073fe0 2024-04-26T19:50:57.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f95a8d33640 1 --2- 172.21.15.79:0/2157411432 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 0x7f95a4073aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f95a9534640 1 -- 172.21.15.79:0/2157411432 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 msgr2=0x7f95a4073aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce45d04640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fce380ead00 0x7fce3804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce45d04640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fce38006ef0 0x7fce38007360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce45d04640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce38053720 con 0x7fce3804ffd0 2024-04-26T19:50:57.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce45d04640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fce380078a0 con 0x7fce38006ef0 2024-04-26T19:50:57.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce45d04640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fce38054330 con 0x7fce380ead00 2024-04-26T19:50:57.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f95a9534640 1 --2- 172.21.15.79:0/2157411432 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 0x7f95a4073aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f95a9534640 1 -- 172.21.15.79:0/2157411432 >> v1:172.21.15.79:6789/0 conn(0x7f95a4073fe0 legacy=0x7f95a40d1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f95a9534640 1 -- 172.21.15.79:0/2157411432 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a4072490 con 0x7f95a40d1930 2024-04-26T19:50:57.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f95a8d33640 1 --2- 172.21.15.79:0/2157411432 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 0x7f95a4073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:57.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce3effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fce3804ffd0 0x7fce38037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:49232/0 (socket says 172.21.15.79:49232) 2024-04-26T19:50:57.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce3effd640 1 -- 172.21.15.79:0/1481346667 learned_addr learned my addr 172.21.15.79:0/1481346667 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:57.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce3ffff640 1 --2- 172.21.15.79:0/1481346667 >> v2:172.21.15.79:3300/0 conn(0x7fce38006ef0 0x7fce38007360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce3e7fc640 1 -- 172.21.15.79:0/1481346667 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 679556434 0 0) 0x7fce38053720 con 0x7fce3804ffd0 2024-04-26T19:50:57.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7fce3e7fc640 1 -- 172.21.15.79:0/1481346667 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce30003520 con 0x7fce3804ffd0 2024-04-26T19:50:57.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f709bfff640 1 --2- 172.21.15.79:0/4097382834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 0x7f709c14d010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f708c009870 tx=0x7f708c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6e40616cd2edb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7fc7c37fe640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b4004800 0x7fc7b4004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7c406bbd0 tx=0x7fc7b4005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b39894097dde6ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f1299893640 1 -- 172.21.15.79:0/2578607459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940b1be0 msgr2=0x7f129408ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7fce3f7fe640 1 --2- 172.21.15.79:0/1481346667 >> v2:172.21.15.83:3300/0 conn(0x7fce380ead00 0x7fce3804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f1299893640 1 --2- 172.21.15.79:0/2578607459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940b1be0 0x7f129408ca60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f12840097a0 tx=0x7f128402a1c0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f1299893640 1 -- 172.21.15.79:0/2578607459 shutdown_connections 2024-04-26T19:50:57.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f1299893640 1 --2- 172.21.15.79:0/2578607459 >> v2:172.21.15.83:3300/0 conn(0x7f1294073ee0 0x7f129416c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f1299893640 1 --2- 172.21.15.79:0/2578607459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940b1be0 0x7f129408ca60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f1299893640 1 -- 172.21.15.79:0/2578607459 >> 172.21.15.79:0/2578607459 conn(0x7f12940b5060 msgr2=0x7f129406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:57.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f1299893640 1 -- 172.21.15.79:0/2578607459 shutdown_connections 2024-04-26T19:50:57.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f1299893640 1 -- 172.21.15.79:0/2578607459 wait complete. 2024-04-26T19:50:57.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7fce3ffff640 1 -- 172.21.15.79:0/1481346667 >> v2:172.21.15.83:3300/0 conn(0x7fce380ead00 msgr2=0x7fce3804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:57.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7fce3ffff640 1 --2- 172.21.15.79:0/1481346667 >> v2:172.21.15.83:3300/0 conn(0x7fce380ead00 0x7fce3804fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7fce3ffff640 1 -- 172.21.15.79:0/1481346667 >> v1:172.21.15.83:6789/0 conn(0x7fce3804ffd0 legacy=0x7fce38037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7fce3e7fc640 1 -- 172.21.15.79:0/1481346667 <== mon.1 v1:172.21.15.83:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1925958614 0 0) 0x7fce30003520 con 0x7fce3804ffd0 2024-04-26T19:50:57.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7fce3ffff640 1 -- 172.21.15.79:0/1481346667 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce38053a00 con 0x7fce38006ef0 2024-04-26T19:50:57.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40b4070 con 0x7fc7b4004800 2024-04-26T19:50:57.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c40a78c0 con 0x7fc7b4004800 2024-04-26T19:50:57.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7fc7c2ffd640 1 -- 172.21.15.79:0/4269013477 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4069390 con 0x7fc7b4004800 2024-04-26T19:50:57.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f95a9534640 1 --2- 172.21.15.79:0/2157411432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95a40d1930 0x7f95a40b06a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f959c009e50 tx=0x7f959c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc9a365bee67993e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f95a37fe640 1 -- 172.21.15.79:0/2157411432 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c0333f0 con 0x7f95a40d1930 2024-04-26T19:50:57.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f95a37fe640 1 -- 172.21.15.79:0/2157411432 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959c033590 con 0x7f95a40d1930 2024-04-26T19:50:57.708 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f95a37fe640 1 -- 172.21.15.79:0/2157411432 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c033960 con 0x7f95a40d1930 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.690+0000 7f8726339640 1 -- start start 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f8726339640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8720071f20 0x7f8720169670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f8726339640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f872016f230 0x7f872008b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f8726339640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 0x7f8720166d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f8726339640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f872008c930 con 0x7f872016f610 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f8726339640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f872008c7b0 con 0x7f872016f230 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f8726339640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f872008cab0 con 0x7f8720071f20 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f871ffff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8720071f20 0x7f8720169670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f87248d6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 0x7f8720166d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f871ffff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8720071f20 0x7f8720169670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:46566/0 (socket says 172.21.15.79:46566) 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f871ffff640 1 -- 172.21.15.79:0/4098016661 learned_addr learned my addr 172.21.15.79:0/4098016661 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f871ffff640 1 -- 172.21.15.79:0/4098016661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f872016f230 msgr2=0x7f872008b9c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T19:50:57.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f871ffff640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f872016f230 0x7f872008b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f871ffff640 1 -- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 msgr2=0x7f8720166d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f871ffff640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 0x7f8720166d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f871ffff640 1 -- 172.21.15.79:0/4098016661 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f872015b560 con 0x7f8720071f20 2024-04-26T19:50:57.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f87248d6640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 0x7f8720166d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:57.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f709a7fc640 1 -- 172.21.15.79:0/4097382834 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c037070 con 0x7f709c0736f0 2024-04-26T19:50:57.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7f709a7fc640 1 -- 172.21.15.79:0/4097382834 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f708c032440 con 0x7f709c0736f0 2024-04-26T19:50:57.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7efdb7fff640 1 -- 172.21.15.79:0/3615469766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb8073fe0 msgr2=0x7efdb80d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7efdb7fff640 1 --2- 172.21.15.79:0/3615469766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb8073fe0 0x7efdb80d1220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efdb8073310 tx=0x7efdb002a710 comp rx=0 tx=0).stop 2024-04-26T19:50:57.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7efdb7fff640 1 -- 172.21.15.79:0/3615469766 shutdown_connections 2024-04-26T19:50:57.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7efdb7fff640 1 --2- 172.21.15.79:0/3615469766 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb8073fe0 0x7efdb80d1220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7efdb7fff640 1 --2- 172.21.15.79:0/3615469766 >> v2:172.21.15.79:3300/0 conn(0x7efdb815b1e0 0x7efdb8073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.691+0000 7efdb7fff640 1 -- 172.21.15.79:0/3615469766 >> 172.21.15.79:0/3615469766 conn(0x7efdb80b4e80 msgr2=0x7efdb806feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:57.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f871ffff640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8720071f20 0x7f8720169670 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f871400b450 tx=0x7f871400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f709a7fc640 1 -- 172.21.15.79:0/4097382834 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c0327f0 con 0x7f709c0736f0 2024-04-26T19:50:57.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f871cff9640 1 -- 172.21.15.79:0/4098016661 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714015020 con 0x7f8720071f20 2024-04-26T19:50:57.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f871cff9640 1 -- 172.21.15.79:0/4098016661 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87140118b0 con 0x7f8720071f20 2024-04-26T19:50:57.712 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f871cff9640 1 -- 172.21.15.79:0/4098016661 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8714011ca0 con 0x7f8720071f20 2024-04-26T19:50:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f70a1cfc640 1 -- 172.21.15.79:0/4097382834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 msgr2=0x7f709c14d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f70a1cfc640 1 --2- 172.21.15.79:0/4097382834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 0x7f709c14d010 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f708c009870 tx=0x7f708c02a290 comp rx=0 tx=0).stop 2024-04-26T19:50:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8720164f90 con 0x7f8720071f20 2024-04-26T19:50:57.713 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8720165550 con 0x7f8720071f20 2024-04-26T19:50:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f95a9d35640 1 -- 172.21.15.79:0/2157411432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95a40d1930 msgr2=0x7f95a40b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7f95a9d35640 1 --2- 172.21.15.79:0/2157411432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95a40d1930 0x7f95a40b06a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f959c009e50 tx=0x7f959c02c8e0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7fc7ca97f640 1 -- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b4004800 msgr2=0x7fc7b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7fc7ca97f640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b4004800 0x7fc7b4004c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc7c406bbd0 tx=0x7fc7b4005830 comp rx=0 tx=0).stop 2024-04-26T19:50:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7fc7ca97f640 1 -- 172.21.15.79:0/4269013477 shutdown_connections 2024-04-26T19:50:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7fc7ca97f640 1 --2- 172.21.15.79:0/4269013477 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7b4004800 0x7fc7b4004c80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7fc7ca97f640 1 -- 172.21.15.79:0/4269013477 >> 172.21.15.79:0/4269013477 conn(0x7fc7bc009fb0 msgr2=0x7fc7bc00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7fc7ca97f640 1 -- 172.21.15.79:0/4269013477 shutdown_connections 2024-04-26T19:50:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7f95a9d35640 1 -- 172.21.15.79:0/2157411432 shutdown_connections 2024-04-26T19:50:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7f95a9d35640 1 --2- 172.21.15.79:0/2157411432 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95a40d1930 0x7f95a40b06a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7f95a9d35640 1 --2- 172.21.15.79:0/2157411432 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 0x7f95a4073aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7f95a9d35640 1 -- 172.21.15.79:0/2157411432 >> 172.21.15.79:0/2157411432 conn(0x7f95a40b4e80 msgr2=0x7f95a406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:57.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.692+0000 7efdb7fff640 1 -- 172.21.15.79:0/3615469766 shutdown_connections 2024-04-26T19:50:57.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7f1299893640 1 Processor -- start 2024-04-26T19:50:57.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7efdb7fff640 1 -- 172.21.15.79:0/3615469766 wait complete. 2024-04-26T19:50:57.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7f95a9d35640 1 -- 172.21.15.79:0/2157411432 shutdown_connections 2024-04-26T19:50:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7fce3ffff640 1 --2- 172.21.15.79:0/1481346667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce38006ef0 0x7fce38007360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fce40051930 tx=0x7fce400a6330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dae590b4c1a21d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7fce3e7fc640 1 -- 172.21.15.79:0/1481346667 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c001670 con 0x7fce38006ef0 2024-04-26T19:50:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7fce3e7fc640 1 -- 172.21.15.79:0/1481346667 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce400aa070 con 0x7fce38006ef0 2024-04-26T19:50:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7fce3e7fc640 1 -- 172.21.15.79:0/1481346667 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce400ae410 con 0x7fce38006ef0 2024-04-26T19:50:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7fc7ca97f640 1 -- 172.21.15.79:0/4269013477 wait complete. 2024-04-26T19:50:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f70a1cfc640 1 -- 172.21.15.79:0/4097382834 shutdown_connections 2024-04-26T19:50:57.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f70a1cfc640 1 --2- 172.21.15.79:0/4097382834 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 0x7f709c14d010 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f70a1cfc640 1 --2- 172.21.15.79:0/4097382834 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f709c0d1500 0x7f709c14cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f70a1cfc640 1 -- 172.21.15.79:0/4097382834 >> 172.21.15.79:0/4097382834 conn(0x7f709c0b4e80 msgr2=0x7f709c06feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:57.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87200738a0 con 0x7f8720071f20 2024-04-26T19:50:57.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7f95a9d35640 1 -- 172.21.15.79:0/2157411432 wait complete. 2024-04-26T19:50:57.718 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f95a9d35640 1 Processor -- start 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.693+0000 7f1299893640 1 -- start start 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f1299893640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1294073ee0 0x7f129417c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f1299893640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12940b1be0 0x7f129415de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f1299893640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940cdad0 0x7f12941559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f1299893640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f12940ceac0 con 0x7f1294073ee0 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f1299893640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f12940cdf30 con 0x7f12940b1be0 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f1299893640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f12940cefe0 con 0x7f12940cdad0 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f12927fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12940b1be0 0x7f129415de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f12927fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12940b1be0 0x7f129415de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:36248/0 (socket says 172.21.15.79:36248) 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f12927fc640 1 -- 172.21.15.79:0/3447559439 learned_addr learned my addr 172.21.15.79:0/3447559439 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f1292ffd640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1294073ee0 0x7f129417c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f12937fe640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940cdad0 0x7f12941559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f12927fc640 1 -- 172.21.15.79:0/3447559439 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940cdad0 msgr2=0x7f12941559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f12927fc640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940cdad0 0x7f12941559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f12927fc640 1 -- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1294073ee0 msgr2=0x7f129417c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f12927fc640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1294073ee0 0x7f129417c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f12927fc640 1 -- 172.21.15.79:0/3447559439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129415e580 con 0x7f12940b1be0 2024-04-26T19:50:57.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f70a1cfc640 1 -- 172.21.15.79:0/4097382834 shutdown_connections 2024-04-26T19:50:57.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f12927fc640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12940b1be0 0x7f129415de10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f128c00cbc0 tx=0x7f128c00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f1292ffd640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1294073ee0 0x7f129417c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:57.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7fce45d04640 1 -- 172.21.15.79:0/1481346667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce38006ef0 msgr2=0x7fce38007360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7fce45d04640 1 --2- 172.21.15.79:0/1481346667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce38006ef0 0x7fce38007360 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fce40051930 tx=0x7fce400a6330 comp rx=0 tx=0).stop 2024-04-26T19:50:57.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7efdb7fff640 1 Processor -- start 2024-04-26T19:50:57.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7fce45d04640 1 -- 172.21.15.79:0/1481346667 shutdown_connections 2024-04-26T19:50:57.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7fce45d04640 1 --2- 172.21.15.79:0/1481346667 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce38006ef0 0x7fce38007360 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7fce45d04640 1 --2- 172.21.15.79:0/1481346667 >> v2:172.21.15.83:3300/0 conn(0x7fce380ead00 0x7fce3804fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7fce45d04640 1 -- 172.21.15.79:0/1481346667 >> 172.21.15.79:0/1481346667 conn(0x7fce38009fb0 msgr2=0x7fce3800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7fc7ca97f640 1 Processor -- start 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7fc7ca97f640 1 -- start start 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7fc7ca97f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b4004800 0x7fc7bc0e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7fc7ca97f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc7bc0d3e20 0x7fc7bc04cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7fc7ca97f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7bc0e9c60 0x7fc7bc0ea110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7fc7ca97f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc054940 con 0x7fc7b4004800 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7fc7ca97f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc7bc0547c0 con 0x7fc7bc0e9c60 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7fc7ca97f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc7bc04f550 con 0x7fc7bc0d3e20 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7fc7c3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b4004800 0x7fc7bc0e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.722 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7fc7c3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b4004800 0x7fc7bc0e5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46180/0 (socket says 172.21.15.79:46180) 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7fc7c3fff640 1 -- 172.21.15.79:0/3696040080 learned_addr learned my addr 172.21.15.79:0/3696040080 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f95a9d35640 1 -- start start 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f95a9d35640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4073fe0 0x7f95a41f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f95a9d35640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95a40d1930 0x7f95a4155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f95a8d33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4073fe0 0x7f95a41f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f95a8d33640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4073fe0 0x7f95a41f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:36260/0 (socket says 172.21.15.79:36260) 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f95a8d33640 1 -- 172.21.15.79:0/1562578607 learned_addr learned my addr 172.21.15.79:0/1562578607 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f95a9d35640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 0x7f95a4155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f95a40719d0 con 0x7f95a40d1930 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f95a4071850 con 0x7f95a4073fe0 2024-04-26T19:50:57.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f95a415b5c0 con 0x7f95a415b1e0 2024-04-26T19:50:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f871cff9640 1 -- 172.21.15.79:0/4098016661 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8714018020 con 0x7f8720071f20 2024-04-26T19:50:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f871cff9640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f870c06c940 0x7f870c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f871f7fe640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f870c06c940 0x7f870c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f871cff9640 1 -- 172.21.15.79:0/4098016661 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f871408c000 con 0x7f8720071f20 2024-04-26T19:50:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.694+0000 7f70a1cfc640 1 -- 172.21.15.79:0/4097382834 wait complete. 2024-04-26T19:50:57.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7efdb7fff640 1 -- start start 2024-04-26T19:50:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7efdb7fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdb8073fe0 0x7efdb814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7efdb7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb80d1760 0x7efdb8155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7efdb7fff640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb815b1e0 0x7efdb814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7efdb7fff640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8071850 con 0x7efdb80d1760 2024-04-26T19:50:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7efdb7fff640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efdb8072770 con 0x7efdb815b1e0 2024-04-26T19:50:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7efdb7fff640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7efdb80719d0 con 0x7efdb8073fe0 2024-04-26T19:50:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f95a3fff640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95a40d1930 0x7f95a4155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f127bfff640 1 -- 172.21.15.79:0/3447559439 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c039070 con 0x7f12940b1be0 2024-04-26T19:50:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f127bfff640 1 -- 172.21.15.79:0/3447559439 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128c00aaa0 con 0x7f12940b1be0 2024-04-26T19:50:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.695+0000 7f127bfff640 1 -- 172.21.15.79:0/3447559439 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c00ae50 con 0x7f12940b1be0 2024-04-26T19:50:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129415e860 con 0x7f12940b1be0 2024-04-26T19:50:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.696+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129417b190 con 0x7f12940b1be0 2024-04-26T19:50:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb67fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb80d1760 0x7efdb8155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.726 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7fc7c8f1c640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7bc0e9c60 0x7fc7bc0ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb67fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb80d1760 0x7efdb8155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46198/0 (socket says 172.21.15.79:46198) 2024-04-26T19:50:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb67fc640 1 -- 172.21.15.79:0/243036845 learned_addr learned my addr 172.21.15.79:0/243036845 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7fce45d04640 1 -- 172.21.15.79:0/1481346667 shutdown_connections 2024-04-26T19:50:57.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f70a1cfc640 1 Processor -- start 2024-04-26T19:50:57.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb6ffd640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdb8073fe0 0x7efdb814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb77fe640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb815b1e0 0x7efdb814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f70a1cfc640 1 -- start start 2024-04-26T19:50:57.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f95a9534640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 0x7f95a4155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f95a8d33640 1 -- 172.21.15.79:0/1562578607 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 msgr2=0x7f95a4155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f95a8d33640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 0x7f95a4155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f95a8d33640 1 -- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95a40d1930 msgr2=0x7f95a4155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f95a8d33640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95a40d1930 0x7f95a4155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7f95a8d33640 1 -- 172.21.15.79:0/1562578607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a4201b10 con 0x7f95a4073fe0 2024-04-26T19:50:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb67fc640 1 -- 172.21.15.79:0/243036845 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdb8073fe0 msgr2=0x7efdb814b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb67fc640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdb8073fe0 0x7efdb814b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb67fc640 1 -- 172.21.15.79:0/243036845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb815b1e0 msgr2=0x7efdb814ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb67fc640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb815b1e0 0x7efdb814ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7efdb67fc640 1 -- 172.21.15.79:0/243036845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdb814b060 con 0x7efdb80d1760 2024-04-26T19:50:57.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7fc7c3fff640 1 -- 172.21.15.79:0/3696040080 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc7bc0d3e20 msgr2=0x7fc7bc04cd10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T19:50:57.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7fc7c3fff640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc7bc0d3e20 0x7fc7bc04cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7fc7c3fff640 1 -- 172.21.15.79:0/3696040080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7bc0e9c60 msgr2=0x7fc7bc0ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7fc7c3fff640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7bc0e9c60 0x7fc7bc0ea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7fc7c3fff640 1 -- 172.21.15.79:0/3696040080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7bc0d43e0 con 0x7fc7b4004800 2024-04-26T19:50:57.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7fc7c3fff640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b4004800 0x7fc7bc0e5f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80016d0 tx=0x7fc7b8001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7f95a8d33640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4073fe0 0x7f95a41f5680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f959800c7f0 tx=0x7f959800ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.697+0000 7fce45d04640 1 -- 172.21.15.79:0/1481346667 wait complete. 2024-04-26T19:50:57.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7efdb67fc640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb80d1760 0x7efdb8155dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efdb0000f80 tx=0x7efdb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7fce45d04640 1 Processor -- start 2024-04-26T19:50:57.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7fce45d04640 1 -- start start 2024-04-26T19:50:57.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7fc7c0ff9640 1 -- 172.21.15.79:0/3696040080 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b801a070 con 0x7fc7b4004800 2024-04-26T19:50:57.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7bc0d9d30 con 0x7fc7b4004800 2024-04-26T19:50:57.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7bc0da2f0 con 0x7fc7b4004800 2024-04-26T19:50:57.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7fc7c0ff9640 1 -- 172.21.15.79:0/3696040080 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b800f750 con 0x7fc7b4004800 2024-04-26T19:50:57.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7fc7c0ff9640 1 -- 172.21.15.79:0/3696040080 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b800fb40 con 0x7fc7b4004800 2024-04-26T19:50:57.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7f95a17fa640 1 -- 172.21.15.79:0/1562578607 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598015070 con 0x7f95a4073fe0 2024-04-26T19:50:57.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7f95a17fa640 1 -- 172.21.15.79:0/1562578607 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9598007820 con 0x7f95a4073fe0 2024-04-26T19:50:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129417b390 con 0x7f12940b1be0 2024-04-26T19:50:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f127bfff640 1 -- 172.21.15.79:0/3447559439 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f128c03c070 con 0x7f12940b1be0 2024-04-26T19:50:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f70a1cfc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 0x7f709c202680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f70a1cfc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f709c0d1500 0x7f709c1f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f709b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 0x7f709c202680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f709b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 0x7f709c202680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46214/0 (socket says 172.21.15.79:46214) 2024-04-26T19:50:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f709affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f709c0d1500 0x7f709c1f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.733 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7efd9f7fe640 1 -- 172.21.15.79:0/243036845 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0041020 con 0x7efdb80d1760 2024-04-26T19:50:57.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7efd9f7fe640 1 -- 172.21.15.79:0/243036845 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdb00087b0 con 0x7efdb80d1760 2024-04-26T19:50:57.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.698+0000 7efd9f7fe640 1 -- 172.21.15.79:0/243036845 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdb0036040 con 0x7efdb80d1760 2024-04-26T19:50:57.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f709affd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f709c0d1500 0x7f709c1f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:36278/0 (socket says 172.21.15.79:36278) 2024-04-26T19:50:57.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f709b7fe640 1 -- 172.21.15.79:0/3046306613 learned_addr learned my addr 172.21.15.79:0/3046306613 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f70a1cfc640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f709c0d18e0 0x7f709c1474e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.734 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f70a1cfc640 1 -- 172.21.15.79:0/3046306613 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f709c163de0 con 0x7f709c0736f0 2024-04-26T19:50:57.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f871f7fe640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f870c06c940 0x7f870c06ee00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f872016d360 tx=0x7f8710008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f70a1cfc640 1 -- 172.21.15.79:0/3046306613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f709c163c60 con 0x7f709c0d1500 2024-04-26T19:50:57.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f709bfff640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f709c0d18e0 0x7f709c1474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f70a1cfc640 1 -- 172.21.15.79:0/3046306613 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f709c0b0d70 con 0x7f709c0d18e0 2024-04-26T19:50:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f127bfff640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f128006ca90 0x7f128006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f127bfff640 1 -- 172.21.15.79:0/3447559439 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f128c0ad960 con 0x7f12940b1be0 2024-04-26T19:50:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.700+0000 7f1292ffd640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f128006ca90 0x7f128006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.700+0000 7f709affd640 1 -- 172.21.15.79:0/3046306613 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f709c0d18e0 msgr2=0x7f709c1474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.700+0000 7f709affd640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f709c0d18e0 0x7f709c1474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.700+0000 7f709affd640 1 -- 172.21.15.79:0/3046306613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 msgr2=0x7f709c202680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.700+0000 7f709affd640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 0x7f709c202680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.700+0000 7f709affd640 1 -- 172.21.15.79:0/3046306613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709c165530 con 0x7f709c0d1500 2024-04-26T19:50:57.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a41513e0 con 0x7f95a4073fe0 2024-04-26T19:50:57.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.700+0000 7f709affd640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f709c0d1500 0x7f709c1f84c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f709400c700 tx=0x7f709400cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a4201d80 con 0x7f95a4073fe0 2024-04-26T19:50:57.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.699+0000 7f95a17fa640 1 -- 172.21.15.79:0/1562578607 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598007bd0 con 0x7f95a4073fe0 2024-04-26T19:50:57.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.700+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdb8156e80 con 0x7efdb80d1760 2024-04-26T19:50:57.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.700+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdb81555d0 con 0x7efdb80d1760 2024-04-26T19:50:57.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fc7c0ff9640 1 -- 172.21.15.79:0/3696040080 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc7b801d070 con 0x7fc7b4004800 2024-04-26T19:50:57.738 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a4050f10 con 0x7f95a4073fe0 2024-04-26T19:50:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7a4005290 con 0x7fc7b4004800 2024-04-26T19:50:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce45d04640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fce38006ef0 0x7fce3804b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce45d04640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce3804ffd0 0x7fce380df170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7f1292ffd640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f128006ca90 0x7f128006ef50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1284034000 tx=0x7f128402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce45d04640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce380ead00 0x7fce3804dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce45d04640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fce380547c0 con 0x7fce380ead00 2024-04-26T19:50:57.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce45d04640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fce38053fc0 con 0x7fce3804ffd0 2024-04-26T19:50:57.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce45d04640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fce38054940 con 0x7fce38006ef0 2024-04-26T19:50:57.740 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fc7c0ff9640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc7a806c9f0 0x7fc7a806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fc7c0ff9640 1 -- 172.21.15.79:0/3696040080 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc7b808b700 con 0x7fc7b4004800 2024-04-26T19:50:57.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce3804ffd0 0x7fce380df170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce3f7fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fce38006ef0 0x7fce3804b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce3effd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce3804ffd0 0x7fce380df170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:36288/0 (socket says 172.21.15.79:36288) 2024-04-26T19:50:57.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce3effd640 1 -- 172.21.15.79:0/2244164168 learned_addr learned my addr 172.21.15.79:0/2244164168 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:57.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fce3f7fe640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fce38006ef0 0x7fce3804b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:46630/0 (socket says 172.21.15.79:46630) 2024-04-26T19:50:57.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.701+0000 7fc7c37fe640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc7a806c9f0 0x7fc7a806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.702+0000 7fce3ffff640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce380ead00 0x7fce3804dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.702+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdb8050f10 con 0x7efdb80d1760 2024-04-26T19:50:57.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.702+0000 7fce3effd640 1 -- 172.21.15.79:0/2244164168 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fce38006ef0 msgr2=0x7fce3804b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.702+0000 7fce3effd640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fce38006ef0 0x7fce3804b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.702+0000 7fce3effd640 1 -- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce380ead00 msgr2=0x7fce3804dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:57.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.702+0000 7fce3effd640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce380ead00 0x7fce3804dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:57.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.702+0000 7fce3effd640 1 -- 172.21.15.79:0/2244164168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce38180b80 con 0x7fce3804ffd0 2024-04-26T19:50:57.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.702+0000 7fc7c37fe640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc7a806c9f0 0x7fc7a806eeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400c1b0 tx=0x7fc7c40ac000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.702+0000 7fce3effd640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce3804ffd0 0x7fce380df170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c750 tx=0x7fce2c00cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.703+0000 7efd9f7fe640 1 -- 172.21.15.79:0/243036845 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdb0044020 con 0x7efdb80d1760 2024-04-26T19:50:57.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.703+0000 7efd9f7fe640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efda406c940 0x7efda406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.704+0000 7efd9f7fe640 1 -- 172.21.15.79:0/243036845 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efdb00ac9d0 con 0x7efdb80d1760 2024-04-26T19:50:57.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.704+0000 7efdb6ffd640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efda406c940 0x7efda406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.704+0000 7efdb6ffd640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efda406c940 0x7efda406ee00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efdb8159c00 tx=0x7efda8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.704+0000 7f871cff9640 1 -- 172.21.15.79:0/4098016661 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f87200738a0 con 0x7f8720071f20 2024-04-26T19:50:57.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7fce2bfff640 1 -- 172.21.15.79:0/2244164168 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c017070 con 0x7fce3804ffd0 2024-04-26T19:50:57.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7fc7c0ff9640 1 -- 172.21.15.79:0/3696040080 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc7b80597d0 con 0x7fc7b4004800 2024-04-26T19:50:57.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7f127bfff640 1 -- 172.21.15.79:0/3447559439 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f129417b390 con 0x7f12940b1be0 2024-04-26T19:50:57.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7fce2bfff640 1 -- 172.21.15.79:0/2244164168 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce2c005ad0 con 0x7fce3804ffd0 2024-04-26T19:50:57.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7fce2bfff640 1 -- 172.21.15.79:0/2244164168 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2c007070 con 0x7fce3804ffd0 2024-04-26T19:50:57.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7f7083fff640 1 -- 172.21.15.79:0/3046306613 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7094003b70 con 0x7f709c0d1500 2024-04-26T19:50:57.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7f70a1cfc640 1 -- 172.21.15.79:0/3046306613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709c165810 con 0x7f709c0d1500 2024-04-26T19:50:57.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7f70a1cfc640 1 -- 172.21.15.79:0/3046306613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709c165dd0 con 0x7f709c0d1500 2024-04-26T19:50:57.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7fce45d04640 1 -- 172.21.15.79:0/2244164168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce38180e60 con 0x7fce3804ffd0 2024-04-26T19:50:57.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.707+0000 7fce45d04640 1 -- 172.21.15.79:0/2244164168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce38181420 con 0x7fce3804ffd0 2024-04-26T19:50:57.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.708+0000 7f7083fff640 1 -- 172.21.15.79:0/3046306613 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7094003d10 con 0x7f709c0d1500 2024-04-26T19:50:57.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.709+0000 7fce29ffb640 1 -- 172.21.15.79:0/2244164168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce380052c0 con 0x7fce3804ffd0 2024-04-26T19:50:57.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.709+0000 7f7083fff640 1 -- 172.21.15.79:0/3046306613 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70940052f0 con 0x7f709c0d1500 2024-04-26T19:50:57.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.710+0000 7f7081ffb640 1 -- 172.21.15.79:0/3046306613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f709c050f10 con 0x7f709c0d1500 2024-04-26T19:50:57.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.711+0000 7f95a17fa640 1 -- 172.21.15.79:0/1562578607 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9598018070 con 0x7f95a4073fe0 2024-04-26T19:50:57.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.711+0000 7f95a17fa640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f959006c9c0 0x7f959006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.712+0000 7f95a3fff640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f959006c9c0 0x7f959006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.712+0000 7f95a17fa640 1 -- 172.21.15.79:0/1562578607 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f959808c2f0 con 0x7f95a4073fe0 2024-04-26T19:50:57.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.712+0000 7f95a3fff640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f959006c9c0 0x7f959006ee80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f95a406fb00 tx=0x7f9594009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.712+0000 7f95a17fa640 1 -- 172.21.15.79:0/1562578607 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f959808c660 con 0x7f95a4073fe0 2024-04-26T19:50:57.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.713+0000 7efd9f7fe640 1 -- 172.21.15.79:0/243036845 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdb004b030 con 0x7efdb80d1760 2024-04-26T19:50:57.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.714+0000 7fce2bfff640 1 -- 172.21.15.79:0/2244164168 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fce2c01a070 con 0x7fce3804ffd0 2024-04-26T19:50:57.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.714+0000 7fce2bfff640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fce2406c8f0 0x7fce2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.715+0000 7fce2bfff640 1 -- 172.21.15.79:0/2244164168 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fce2c08b830 con 0x7fce3804ffd0 2024-04-26T19:50:57.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.715+0000 7fce3f7fe640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fce2406c8f0 0x7fce2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.714+0000 7f7083fff640 1 -- 172.21.15.79:0/3046306613 <== mon.1 v2:172.21.15.83:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7094005490 con 0x7f709c0d1500 2024-04-26T19:50:57.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.714+0000 7f7083fff640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f708806c9f0 0x7f708806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:57.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.715+0000 7f7083fff640 1 -- 172.21.15.79:0/3046306613 <== mon.1 v2:172.21.15.83:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f709408c640 con 0x7f709c0d1500 2024-04-26T19:50:57.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.716+0000 7fce3f7fe640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fce2406c8f0 0x7fce2406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fce34009a30 tx=0x7fce34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.715+0000 7f709b7fe640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f708806c9f0 0x7f708806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:57.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.716+0000 7f709b7fe640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f708806c9f0 0x7f708806eeb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f709c0cd500 tx=0x7f7090009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:57.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.718+0000 7fce2bfff640 1 -- 172.21.15.79:0/2244164168 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fce2c021020 con 0x7fce3804ffd0 2024-04-26T19:50:57.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:57.722+0000 7f7083fff640 1 -- 172.21.15.79:0/3046306613 <== mon.1 v2:172.21.15.83:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f709405a0c0 con 0x7f709c0d1500 2024-04-26T19:50:58.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.067+0000 7f8ef6425640 1 -- 172.21.15.79:0/3948405157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8ecc005580 con 0x7f8ef00ce6d0 2024-04-26T19:50:58.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.068+0000 7f8eecff9640 1 -- 172.21.15.79:0/3948405157 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8ee408b0e0 con 0x7f8ef00ce6d0 2024-04-26T19:50:58.068 INFO:teuthology.orchestra.run.smithi079.stdout:55834574859 2024-04-26T19:50:58.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.071+0000 7f8ede7fc640 1 -- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8ed406c9c0 msgr2=0x7f8ed406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.071+0000 7f8ede7fc640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8ed406c9c0 0x7f8ed406ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ef0150b80 tx=0x7f8ee0009060 comp rx=0 tx=0).stop 2024-04-26T19:50:58.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.071+0000 7f8ede7fc640 1 -- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef00ce6d0 msgr2=0x7f8ef0150bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.071+0000 7f8ede7fc640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef00ce6d0 0x7f8ef0150bd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400b750 tx=0x7f8ee400bc10 comp rx=0 tx=0).stop 2024-04-26T19:50:58.073 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.073+0000 7f8ede7fc640 1 -- 172.21.15.79:0/3948405157 shutdown_connections 2024-04-26T19:50:58.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.073+0000 7f8ede7fc640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8ed406c9c0 0x7f8ed406ee80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.073+0000 7f8ede7fc640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8ef0140b90 0x7f8ef013e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.073+0000 7f8ede7fc640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8ef013e2a0 0x7f8ef0150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.073+0000 7f8ede7fc640 1 --2- 172.21.15.79:0/3948405157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8ef00ce6d0 0x7f8ef0150bd0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.073+0000 7f8ede7fc640 1 -- 172.21.15.79:0/3948405157 >> 172.21.15.79:0/3948405157 conn(0x7f8ef00b2e00 msgr2=0x7f8ef00b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.073+0000 7f8ede7fc640 1 -- 172.21.15.79:0/3948405157 shutdown_connections 2024-04-26T19:50:58.074 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.073+0000 7f8ede7fc640 1 -- 172.21.15.79:0/3948405157 wait complete. 2024-04-26T19:50:58.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-26T19:50:58.091 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:58.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.104+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7efdb815a580 con 0x7efdb80d1760 2024-04-26T19:50:58.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.104+0000 7efd9f7fe640 1 -- 172.21.15.79:0/243036845 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efdb007aba0 con 0x7efdb80d1760 2024-04-26T19:50:58.105 INFO:teuthology.orchestra.run.smithi079.stdout:55834574858 2024-04-26T19:50:58.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.106+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efda406c940 msgr2=0x7efda406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.106+0000 7efdb7fff640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efda406c940 0x7efda406ee00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efdb8159c00 tx=0x7efda8008040 comp rx=0 tx=0).stop 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.106+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb80d1760 msgr2=0x7efdb8155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.106+0000 7efdb7fff640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb80d1760 0x7efdb8155dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7efdb0000f80 tx=0x7efdb0004a40 comp rx=0 tx=0).stop 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.106+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 shutdown_connections 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.106+0000 7efdb7fff640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efda406c940 0x7efda406ee00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.107+0000 7efdb7fff640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdb815b1e0 0x7efdb814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.107+0000 7efdb7fff640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdb80d1760 0x7efdb8155dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.107+0000 7efdb7fff640 1 --2- 172.21.15.79:0/243036845 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdb8073fe0 0x7efdb814b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.107+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 >> 172.21.15.79:0/243036845 conn(0x7efdb80b4e80 msgr2=0x7efdb815f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.107+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 shutdown_connections 2024-04-26T19:50:58.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.107+0000 7efdb7fff640 1 -- 172.21.15.79:0/243036845 wait complete. 2024-04-26T19:50:58.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.114+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f8720050f10 con 0x7f8720071f20 2024-04-26T19:50:58.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.115+0000 7f871cff9640 1 -- 172.21.15.79:0/4098016661 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f871401f020 con 0x7f8720071f20 2024-04-26T19:50:58.117 INFO:teuthology.orchestra.run.smithi079.stdout:55834574859 2024-04-26T19:50:58.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-26T19:50:58.119 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:58.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f870c06c940 msgr2=0x7f870c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f870c06c940 0x7f870c06ee00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f872016d360 tx=0x7f8710008040 comp rx=0 tx=0).stop 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8720071f20 msgr2=0x7f8720169670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8720071f20 0x7f8720169670 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f871400b450 tx=0x7f871400b910 comp rx=0 tx=0).stop 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 shutdown_connections 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f870c06c940 0x7f870c06ee00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f872016f610 0x7f8720166d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f872016f230 0x7f872008b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 --2- 172.21.15.79:0/4098016661 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8720071f20 0x7f8720169670 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f871400b450 tx=0x7f871400b910 comp rx=0 tx=0).stop 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 >> 172.21.15.79:0/4098016661 conn(0x7f87200b4e80 msgr2=0x7f872016aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 shutdown_connections 2024-04-26T19:50:58.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.119+0000 7f8726339640 1 -- 172.21.15.79:0/4098016661 wait complete. 2024-04-26T19:50:58.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.125+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f12940506c0 con 0x7f12940b1be0 2024-04-26T19:50:58.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.126+0000 7f127bfff640 1 -- 172.21.15.79:0/3447559439 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f128c07b400 con 0x7f12940b1be0 2024-04-26T19:50:58.126 INFO:teuthology.orchestra.run.smithi079.stdout:55834574860 2024-04-26T19:50:58.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.127+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc7a4005580 con 0x7fc7b4004800 2024-04-26T19:50:58.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7fc7c0ff9640 1 -- 172.21.15.79:0/3696040080 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7b80590d0 con 0x7fc7b4004800 2024-04-26T19:50:58.129 INFO:teuthology.orchestra.run.smithi079.stdout:55834574858 2024-04-26T19:50:58.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f128006ca90 msgr2=0x7f128006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f128006ca90 0x7f128006ef50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1284034000 tx=0x7f128402e000 comp rx=0 tx=0).stop 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12940b1be0 msgr2=0x7f129415de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12940b1be0 0x7f129415de10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f128c00cbc0 tx=0x7f128c00a060 comp rx=0 tx=0).stop 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 shutdown_connections 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f128006ca90 0x7f128006ef50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f12940cdad0 0x7f12941559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f12940b1be0 0x7f129415de10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 --2- 172.21.15.79:0/3447559439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1294073ee0 0x7f129417c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 >> 172.21.15.79:0/3447559439 conn(0x7f12940b5060 msgr2=0x7f129415f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 shutdown_connections 2024-04-26T19:50:58.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.128+0000 7f1299893640 1 -- 172.21.15.79:0/3447559439 wait complete. 2024-04-26T19:50:58.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc7a806c9f0 msgr2=0x7fc7a806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc7a806c9f0 0x7fc7a806eeb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400c1b0 tx=0x7fc7c40ac000 comp rx=0 tx=0).stop 2024-04-26T19:50:58.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b4004800 msgr2=0x7fc7bc0e5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b4004800 0x7fc7bc0e5f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80016d0 tx=0x7fc7b8001b90 comp rx=0 tx=0).stop 2024-04-26T19:50:58.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 shutdown_connections 2024-04-26T19:50:58.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc7a806c9f0 0x7fc7a806eeb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc7bc0e9c60 0x7fc7bc0ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc7bc0d3e20 0x7fc7bc04cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 --2- 172.21.15.79:0/3696040080 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc7b4004800 0x7fc7bc0e5f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 >> 172.21.15.79:0/3696040080 conn(0x7fc7bc009fb0 msgr2=0x7fc7bc037b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.130+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 shutdown_connections 2024-04-26T19:50:58.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.132+0000 7fc7ca97f640 1 -- 172.21.15.79:0/3696040080 wait complete. 2024-04-26T19:50:58.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-04-26T19:50:58.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.136+0000 7f7081ffb640 1 -- 172.21.15.79:0/3046306613 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f709c072960 con 0x7f709c0d1500 2024-04-26T19:50:58.136 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:58.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.136+0000 7f7083fff640 1 -- 172.21.15.79:0/3046306613 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f709405e3c0 con 0x7f709c0d1500 2024-04-26T19:50:58.137 INFO:teuthology.orchestra.run.smithi079.stdout:55834574857 2024-04-26T19:50:58.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 -- 172.21.15.79:0/3046306613 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f708806c9f0 msgr2=0x7f708806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f708806c9f0 0x7f708806eeb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f709c0cd500 tx=0x7f7090009190 comp rx=0 tx=0).stop 2024-04-26T19:50:58.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 -- 172.21.15.79:0/3046306613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f709c0d1500 msgr2=0x7f709c1f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f709c0d1500 0x7f709c1f84c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f709400c700 tx=0x7f709400cbc0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 -- 172.21.15.79:0/3046306613 shutdown_connections 2024-04-26T19:50:58.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f708806c9f0 0x7f708806eeb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f709c0d18e0 0x7f709c1474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f709c0d1500 0x7f709c1f84c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 --2- 172.21.15.79:0/3046306613 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f709c0736f0 0x7f709c202680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.138+0000 7f7081ffb640 1 -- 172.21.15.79:0/3046306613 >> 172.21.15.79:0/3046306613 conn(0x7f709c0b4e80 msgr2=0x7f709c14c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.139+0000 7f7081ffb640 1 -- 172.21.15.79:0/3046306613 shutdown_connections 2024-04-26T19:50:58.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.139+0000 7f7081ffb640 1 -- 172.21.15.79:0/3046306613 wait complete. 2024-04-26T19:50:58.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.139+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f95a40703a0 con 0x7f95a4073fe0 2024-04-26T19:50:58.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.139+0000 7f95a17fa640 1 -- 172.21.15.79:0/1562578607 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9598059e00 con 0x7f95a4073fe0 2024-04-26T19:50:58.140 INFO:teuthology.orchestra.run.smithi079.stdout:55834574860 2024-04-26T19:50:58.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2024-04-26T19:50:58.142 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:58.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f959006c9c0 msgr2=0x7f959006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f959006c9c0 0x7f959006ee80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f95a406fb00 tx=0x7f9594009040 comp rx=0 tx=0).stop 2024-04-26T19:50:58.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4073fe0 msgr2=0x7f95a41f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4073fe0 0x7f95a41f5680 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f959800c7f0 tx=0x7f959800ccb0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 shutdown_connections 2024-04-26T19:50:58.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f959006c9c0 0x7f959006ee80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95a415b1e0 0x7f95a4155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95a40d1930 0x7f95a4155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 --2- 172.21.15.79:0/1562578607 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95a4073fe0 0x7f95a41f5680 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.142+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 >> 172.21.15.79:0/1562578607 conn(0x7f95a40b4e80 msgr2=0x7f95a415ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.143+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 shutdown_connections 2024-04-26T19:50:58.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.143+0000 7f95a9d35640 1 -- 172.21.15.79:0/1562578607 wait complete. 2024-04-26T19:50:58.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-26T19:50:58.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.146+0000 7fce29ffb640 1 -- 172.21.15.79:0/2244164168 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fce380372d0 con 0x7fce3804ffd0 2024-04-26T19:50:58.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.146+0000 7fce2bfff640 1 -- 172.21.15.79:0/2244164168 <== mon.1 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fce2c059900 con 0x7fce3804ffd0 2024-04-26T19:50:58.148 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:58.149 INFO:teuthology.orchestra.run.smithi079.stdout:55834574858 2024-04-26T19:50:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 -- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fce2406c8f0 msgr2=0x7fce2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fce2406c8f0 0x7fce2406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fce34009a30 tx=0x7fce34008040 comp rx=0 tx=0).stop 2024-04-26T19:50:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 -- 172.21.15.79:0/2244164168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce3804ffd0 msgr2=0x7fce380df170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce3804ffd0 0x7fce380df170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fce2c00c750 tx=0x7fce2c00cc10 comp rx=0 tx=0).stop 2024-04-26T19:50:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 -- 172.21.15.79:0/2244164168 shutdown_connections 2024-04-26T19:50:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fce2406c8f0 0x7fce2406edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fce380ead00 0x7fce3804dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fce3804ffd0 0x7fce380df170 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 --2- 172.21.15.79:0/2244164168 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fce38006ef0 0x7fce3804b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 -- 172.21.15.79:0/2244164168 >> 172.21.15.79:0/2244164168 conn(0x7fce38009fb0 msgr2=0x7fce38036590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 -- 172.21.15.79:0/2244164168 shutdown_connections 2024-04-26T19:50:58.151 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.149+0000 7fce29ffb640 1 -- 172.21.15.79:0/2244164168 wait complete. 2024-04-26T19:50:58.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.6 2024-04-26T19:50:58.153 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:58.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.1 2024-04-26T19:50:58.156 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:58.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2024-04-26T19:50:58.162 DEBUG:teuthology.parallel:result is None 2024-04-26T19:50:58.162 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T19:50:58.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:50:58.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.298+0000 7f0c75738640 1 Processor -- start 2024-04-26T19:50:58.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.298+0000 7f0c75738640 1 -- start start 2024-04-26T19:50:58.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.299+0000 7f0c75738640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0c70162970 0x7f0c70160440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.299+0000 7f0c75738640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 0x7f0c700b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.299+0000 7f0c75738640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c701634e0 con 0x7f0c700b7ce0 2024-04-26T19:50:58.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.299+0000 7f0c75738640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f0c70164060 con 0x7f0c70162970 2024-04-26T19:50:58.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.299+0000 7f0c75738640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0c701637c0 con 0x7f0c70162010 2024-04-26T19:50:58.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.299+0000 7f0c6effd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0c70162970 0x7f0c70160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.299+0000 7f0c6effd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0c70162970 0x7f0c70160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46228/0 (socket says 172.21.15.79:46228) 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.299+0000 7f0c6effd640 1 -- 172.21.15.79:0/910666769 learned_addr learned my addr 172.21.15.79:0/910666769 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.300+0000 7f0c6f7fe640 1 --2- 172.21.15.79:0/910666769 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 0x7f0c700b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.300+0000 7f0c6effd640 1 -- 172.21.15.79:0/910666769 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 msgr2=0x7f0c700b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.300+0000 7f0c6effd640 1 --2- 172.21.15.79:0/910666769 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 0x7f0c700b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.300+0000 7f0c6effd640 1 -- 172.21.15.79:0/910666769 >> v1:172.21.15.83:6789/0 conn(0x7f0c700b7ce0 legacy=0x7f0c7006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.300+0000 7f0c6effd640 1 -- 172.21.15.79:0/910666769 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c70163aa0 con 0x7f0c70162970 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.300+0000 7f0c6effd640 1 --2- 172.21.15.79:0/910666769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 0x7f0c70160440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009870 tx=0x7f0c6002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd4df5af9d58619 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c6e7fc640 1 -- 172.21.15.79:0/910666769 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60037070 con 0x7f0c70162970 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c6e7fc640 1 -- 172.21.15.79:0/910666769 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c60032440 con 0x7f0c70162970 2024-04-26T19:50:58.302 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c6e7fc640 1 -- 172.21.15.79:0/910666769 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c60032810 con 0x7f0c70162970 2024-04-26T19:50:58.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c75738640 1 -- 172.21.15.79:0/910666769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 msgr2=0x7f0c70160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c75738640 1 --2- 172.21.15.79:0/910666769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 0x7f0c70160440 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0c60009870 tx=0x7f0c6002a1c0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c75738640 1 -- 172.21.15.79:0/910666769 shutdown_connections 2024-04-26T19:50:58.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c75738640 1 --2- 172.21.15.79:0/910666769 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 0x7f0c700b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c75738640 1 --2- 172.21.15.79:0/910666769 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 0x7f0c70160440 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c75738640 1 -- 172.21.15.79:0/910666769 >> 172.21.15.79:0/910666769 conn(0x7f0c700741a0 msgr2=0x7f0c700b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c75738640 1 -- 172.21.15.79:0/910666769 shutdown_connections 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.301+0000 7f0c75738640 1 -- 172.21.15.79:0/910666769 wait complete. 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.302+0000 7f0c75738640 1 Processor -- start 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.302+0000 7f0c75738640 1 -- start start 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c75738640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c700b7ce0 0x7f0c7014bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c75738640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 0x7f0c700d0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c75738640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 0x7f0c700d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c75738640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0c70160980 con 0x7f0c70162970 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c75738640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7006dc10 con 0x7f0c700b7ce0 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c75738640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0c70162d50 con 0x7f0c70162010 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c6f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 0x7f0c700d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c6f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 0x7f0c700d0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46230/0 (socket says 172.21.15.79:46230) 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c6f7fe640 1 -- 172.21.15.79:0/4269769733 learned_addr learned my addr 172.21.15.79:0/4269769733 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:58.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c6effd640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c700b7ce0 0x7f0c7014bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c67fff640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 0x7f0c700d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c6f7fe640 1 -- 172.21.15.79:0/4269769733 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 msgr2=0x7f0c700d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c6f7fe640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 0x7f0c700d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c6f7fe640 1 -- 172.21.15.79:0/4269769733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c700b7ce0 msgr2=0x7f0c7014bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c6f7fe640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c700b7ce0 0x7f0c7014bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.303+0000 7f0c6f7fe640 1 -- 172.21.15.79:0/4269769733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c70157770 con 0x7f0c70162970 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.304+0000 7f0c6f7fe640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 0x7f0c700d0ca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800efc0 tx=0x7f0c6800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.304+0000 7f0c677fe640 1 -- 172.21.15.79:0/4269769733 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68018020 con 0x7f0c70162970 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.304+0000 7f0c677fe640 1 -- 172.21.15.79:0/4269769733 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c68014480 con 0x7f0c70162970 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.304+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c70157a50 con 0x7f0c70162970 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.304+0000 7f0c677fe640 1 -- 172.21.15.79:0/4269769733 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c68014870 con 0x7f0c70162970 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.304+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c70158010 con 0x7f0c70162970 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.305+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c70158210 con 0x7f0c70162970 2024-04-26T19:50:58.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.305+0000 7f0c677fe640 1 -- 172.21.15.79:0/4269769733 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c68005d10 con 0x7f0c70162970 2024-04-26T19:50:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.306+0000 7f0c677fe640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0c5006c8f0 0x7f0c5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.306+0000 7f0c6effd640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0c5006c8f0 0x7f0c5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.306+0000 7f0c677fe640 1 -- 172.21.15.79:0/4269769733 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0c6808c8d0 con 0x7f0c70162970 2024-04-26T19:50:58.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.306+0000 7f0c6effd640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0c5006c8f0 0x7f0c5006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c60034000 tx=0x7f0c6002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:58.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.310+0000 7f0c677fe640 1 -- 172.21.15.79:0/4269769733 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0c68022070 con 0x7f0c70162970 2024-04-26T19:50:58.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.484+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c7006d430 con 0x7f0c5006c8f0 2024-04-26T19:50:58.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.492+0000 7f0c677fe640 1 -- 172.21.15.79:0/4269769733 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7f0c7006d430 con 0x7f0c5006c8f0 2024-04-26T19:50:58.496 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:58.498 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:50:58.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0c5006c8f0 msgr2=0x7f0c5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0c5006c8f0 0x7f0c5006edb0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0c60034000 tx=0x7f0c6002e000 comp rx=0 tx=0).stop 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 msgr2=0x7f0c700d0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 0x7f0c700d0ca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800efc0 tx=0x7f0c6800c490 comp rx=0 tx=0).stop 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 shutdown_connections 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0c5006c8f0 0x7f0c5006edb0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0c70162970 0x7f0c700d0ca0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0c70162010 0x7f0c700d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 --2- 172.21.15.79:0/4269769733 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0c700b7ce0 0x7f0c7014bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 >> 172.21.15.79:0/4269769733 conn(0x7f0c700741a0 msgr2=0x7f0c700ced30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 shutdown_connections 2024-04-26T19:50:58.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.496+0000 7f0c75738640 1 -- 172.21.15.79:0/4269769733 wait complete. 2024-04-26T19:50:58.507 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-26T19:50:58.258498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219460,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755260,"statfs":{"total":773094113280,"available":772869386240,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003297"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300284+0000","last_change":"2024-04-26T19:50:30.105908+0000","last_active":"2024-04-26T19:50:38.300284+0000","last_peered":"2024-04-26T19:50:38.300284+0000","last_clean":"2024-04-26T19:50:38.300284+0000","last_became_active":"2024-04-26T19:50:30.105650+0000","last_became_peered":"2024-04-26T19:50:30.105650+0000","last_unstale":"2024-04-26T19:50:38.300284+0000","last_undegraded":"2024-04-26T19:50:38.300284+0000","last_fullsized":"2024-04-26T19:50:38.300284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:25.896804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131172+0000","last_change":"2024-04-26T19:50:31.110936+0000","last_active":"2024-04-26T19:50:37.131172+0000","last_peered":"2024-04-26T19:50:37.131172+0000","last_clean":"2024-04-26T19:50:37.131172+0000","last_became_active":"2024-04-26T19:50:31.110602+0000","last_became_peered":"2024-04-26T19:50:31.110602+0000","last_unstale":"2024-04-26T19:50:37.131172+0000","last_undegraded":"2024-04-26T19:50:37.131172+0000","last_fullsized":"2024-04-26T19:50:37.131172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130254+0000","last_change":"2024-04-26T19:50:35.626950+0000","last_active":"2024-04-26T19:50:37.130254+0000","last_peered":"2024-04-26T19:50:37.130254+0000","last_clean":"2024-04-26T19:50:37.130254+0000","last_became_active":"2024-04-26T19:50:35.626309+0000","last_became_peered":"2024-04-26T19:50:35.626309+0000","last_unstale":"2024-04-26T19:50:37.130254+0000","last_undegraded":"2024-04-26T19:50:37.130254+0000","last_fullsized":"2024-04-26T19:50:37.130254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:30:42.055139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131089+0000","last_change":"2024-04-26T19:50:30.112742+0000","last_active":"2024-04-26T19:50:37.131089+0000","last_peered":"2024-04-26T19:50:37.131089+0000","last_clean":"2024-04-26T19:50:37.131089+0000","last_became_active":"2024-04-26T19:50:30.112460+0000","last_became_peered":"2024-04-26T19:50:30.112460+0000","last_unstale":"2024-04-26T19:50:37.131089+0000","last_undegraded":"2024-04-26T19:50:37.131089+0000","last_fullsized":"2024-04-26T19:50:37.131089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:21.153340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300125+0000","last_change":"2024-04-26T19:50:31.114135+0000","last_active":"2024-04-26T19:50:38.300125+0000","last_peered":"2024-04-26T19:50:38.300125+0000","last_clean":"2024-04-26T19:50:38.300125+0000","last_became_active":"2024-04-26T19:50:31.113834+0000","last_became_peered":"2024-04-26T19:50:31.113834+0000","last_unstale":"2024-04-26T19:50:38.300125+0000","last_undegraded":"2024-04-26T19:50:38.300125+0000","last_fullsized":"2024-04-26T19:50:38.300125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129662+0000","last_change":"2024-04-26T19:50:35.141805+0000","last_active":"2024-04-26T19:50:37.129662+0000","last_peered":"2024-04-26T19:50:37.129662+0000","last_clean":"2024-04-26T19:50:37.129662+0000","last_became_active":"2024-04-26T19:50:35.141362+0000","last_became_peered":"2024-04-26T19:50:35.141362+0000","last_unstale":"2024-04-26T19:50:37.129662+0000","last_undegraded":"2024-04-26T19:50:37.129662+0000","last_fullsized":"2024-04-26T19:50:37.129662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:10.325286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653474+0000","last_change":"2024-04-26T19:50:30.107616+0000","last_active":"2024-04-26T19:50:38.653474+0000","last_peered":"2024-04-26T19:50:38.653474+0000","last_clean":"2024-04-26T19:50:38.653474+0000","last_became_active":"2024-04-26T19:50:30.107304+0000","last_became_peered":"2024-04-26T19:50:30.107304+0000","last_unstale":"2024-04-26T19:50:38.653474+0000","last_undegraded":"2024-04-26T19:50:38.653474+0000","last_fullsized":"2024-04-26T19:50:38.653474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:12.144175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652574+0000","last_change":"2024-04-26T19:50:31.114725+0000","last_active":"2024-04-26T19:50:38.652574+0000","last_peered":"2024-04-26T19:50:38.652574+0000","last_clean":"2024-04-26T19:50:38.652574+0000","last_became_active":"2024-04-26T19:50:31.114444+0000","last_became_peered":"2024-04-26T19:50:31.114444+0000","last_unstale":"2024-04-26T19:50:38.652574+0000","last_undegraded":"2024-04-26T19:50:38.652574+0000","last_fullsized":"2024-04-26T19:50:38.652574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129888+0000","last_change":"2024-04-26T19:50:35.145133+0000","last_active":"2024-04-26T19:50:37.129888+0000","last_peered":"2024-04-26T19:50:37.129888+0000","last_clean":"2024-04-26T19:50:37.129888+0000","last_became_active":"2024-04-26T19:50:35.144310+0000","last_became_peered":"2024-04-26T19:50:35.144310+0000","last_unstale":"2024-04-26T19:50:37.129888+0000","last_undegraded":"2024-04-26T19:50:37.129888+0000","last_fullsized":"2024-04-26T19:50:37.129888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:35.220332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652453+0000","last_change":"2024-04-26T19:50:30.115029+0000","last_active":"2024-04-26T19:50:38.652453+0000","last_peered":"2024-04-26T19:50:38.652453+0000","last_clean":"2024-04-26T19:50:38.652453+0000","last_became_active":"2024-04-26T19:50:30.114719+0000","last_became_peered":"2024-04-26T19:50:30.114719+0000","last_unstale":"2024-04-26T19:50:38.652453+0000","last_undegraded":"2024-04-26T19:50:38.652453+0000","last_fullsized":"2024-04-26T19:50:38.652453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:28.557299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654620+0000","last_change":"2024-04-26T19:50:31.114802+0000","last_active":"2024-04-26T19:50:38.654620+0000","last_peered":"2024-04-26T19:50:38.654620+0000","last_clean":"2024-04-26T19:50:38.654620+0000","last_became_active":"2024-04-26T19:50:31.114489+0000","last_became_peered":"2024-04-26T19:50:31.114489+0000","last_unstale":"2024-04-26T19:50:38.654620+0000","last_undegraded":"2024-04-26T19:50:38.654620+0000","last_fullsized":"2024-04-26T19:50:38.654620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129602+0000","last_change":"2024-04-26T19:50:35.147238+0000","last_active":"2024-04-26T19:50:37.129602+0000","last_peered":"2024-04-26T19:50:37.129602+0000","last_clean":"2024-04-26T19:50:37.129602+0000","last_became_active":"2024-04-26T19:50:35.146793+0000","last_became_peered":"2024-04-26T19:50:35.146793+0000","last_unstale":"2024-04-26T19:50:37.129602+0000","last_undegraded":"2024-04-26T19:50:37.129602+0000","last_fullsized":"2024-04-26T19:50:37.129602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:14.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654487+0000","last_change":"2024-04-26T19:50:31.104842+0000","last_active":"2024-04-26T19:50:38.654487+0000","last_peered":"2024-04-26T19:50:38.654487+0000","last_clean":"2024-04-26T19:50:38.654487+0000","last_became_active":"2024-04-26T19:50:31.104587+0000","last_became_peered":"2024-04-26T19:50:31.104587+0000","last_unstale":"2024-04-26T19:50:38.654487+0000","last_undegraded":"2024-04-26T19:50:38.654487+0000","last_fullsized":"2024-04-26T19:50:38.654487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653208+0000","last_change":"2024-04-26T19:50:30.112050+0000","last_active":"2024-04-26T19:50:38.653208+0000","last_peered":"2024-04-26T19:50:38.653208+0000","last_clean":"2024-04-26T19:50:38.653208+0000","last_became_active":"2024-04-26T19:50:30.111754+0000","last_became_peered":"2024-04-26T19:50:30.111754+0000","last_unstale":"2024-04-26T19:50:38.653208+0000","last_undegraded":"2024-04-26T19:50:38.653208+0000","last_fullsized":"2024-04-26T19:50:38.653208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130351+0000","last_change":"2024-04-26T19:50:35.145983+0000","last_active":"2024-04-26T19:50:37.130351+0000","last_peered":"2024-04-26T19:50:37.130351+0000","last_clean":"2024-04-26T19:50:37.130351+0000","last_became_active":"2024-04-26T19:50:35.144815+0000","last_became_peered":"2024-04-26T19:50:35.144815+0000","last_unstale":"2024-04-26T19:50:37.130351+0000","last_undegraded":"2024-04-26T19:50:37.130351+0000","last_fullsized":"2024-04-26T19:50:37.130351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130515+0000","last_change":"2024-04-26T19:50:31.109267+0000","last_active":"2024-04-26T19:50:37.130515+0000","last_peered":"2024-04-26T19:50:37.130515+0000","last_clean":"2024-04-26T19:50:37.130515+0000","last_became_active":"2024-04-26T19:50:31.108771+0000","last_became_peered":"2024-04-26T19:50:31.108771+0000","last_unstale":"2024-04-26T19:50:37.130515+0000","last_undegraded":"2024-04-26T19:50:37.130515+0000","last_fullsized":"2024-04-26T19:50:37.130515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997666+0000","last_change":"2024-04-26T19:50:30.110310+0000","last_active":"2024-04-26T19:50:37.997666+0000","last_peered":"2024-04-26T19:50:37.997666+0000","last_clean":"2024-04-26T19:50:37.997666+0000","last_became_active":"2024-04-26T19:50:30.109819+0000","last_became_peered":"2024-04-26T19:50:30.109819+0000","last_unstale":"2024-04-26T19:50:37.997666+0000","last_undegraded":"2024-04-26T19:50:37.997666+0000","last_fullsized":"2024-04-26T19:50:37.997666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:41.838039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130058+0000","last_change":"2024-04-26T19:50:35.143564+0000","last_active":"2024-04-26T19:50:37.130058+0000","last_peered":"2024-04-26T19:50:37.130058+0000","last_clean":"2024-04-26T19:50:37.130058+0000","last_became_active":"2024-04-26T19:50:35.143119+0000","last_became_peered":"2024-04-26T19:50:35.143119+0000","last_unstale":"2024-04-26T19:50:37.130058+0000","last_undegraded":"2024-04-26T19:50:37.130058+0000","last_fullsized":"2024-04-26T19:50:37.130058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:11.052818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130395+0000","last_change":"2024-04-26T19:50:31.112367+0000","last_active":"2024-04-26T19:50:37.130395+0000","last_peered":"2024-04-26T19:50:37.130395+0000","last_clean":"2024-04-26T19:50:37.130395+0000","last_became_active":"2024-04-26T19:50:31.112070+0000","last_became_peered":"2024-04-26T19:50:31.112070+0000","last_unstale":"2024-04-26T19:50:37.130395+0000","last_undegraded":"2024-04-26T19:50:37.130395+0000","last_fullsized":"2024-04-26T19:50:37.130395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997401+0000","last_change":"2024-04-26T19:50:30.105988+0000","last_active":"2024-04-26T19:50:37.997401+0000","last_peered":"2024-04-26T19:50:37.997401+0000","last_clean":"2024-04-26T19:50:37.997401+0000","last_became_active":"2024-04-26T19:50:30.103503+0000","last_became_peered":"2024-04-26T19:50:30.103503+0000","last_unstale":"2024-04-26T19:50:37.997401+0000","last_undegraded":"2024-04-26T19:50:37.997401+0000","last_fullsized":"2024-04-26T19:50:37.997401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997963+0000","last_change":"2024-04-26T19:50:35.626255+0000","last_active":"2024-04-26T19:50:37.997963+0000","last_peered":"2024-04-26T19:50:37.997963+0000","last_clean":"2024-04-26T19:50:37.997963+0000","last_became_active":"2024-04-26T19:50:35.625845+0000","last_became_peered":"2024-04-26T19:50:35.625845+0000","last_unstale":"2024-04-26T19:50:37.997963+0000","last_undegraded":"2024-04-26T19:50:37.997963+0000","last_fullsized":"2024-04-26T19:50:37.997963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:02.874670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129835+0000","last_change":"2024-04-26T19:50:31.106338+0000","last_active":"2024-04-26T19:50:37.129835+0000","last_peered":"2024-04-26T19:50:37.129835+0000","last_clean":"2024-04-26T19:50:37.129835+0000","last_became_active":"2024-04-26T19:50:31.106097+0000","last_became_peered":"2024-04-26T19:50:31.106097+0000","last_unstale":"2024-04-26T19:50:37.129835+0000","last_undegraded":"2024-04-26T19:50:37.129835+0000","last_fullsized":"2024-04-26T19:50:37.129835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:13.140788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654064+0000","last_change":"2024-04-26T19:50:30.114014+0000","last_active":"2024-04-26T19:50:38.654064+0000","last_peered":"2024-04-26T19:50:38.654064+0000","last_clean":"2024-04-26T19:50:38.654064+0000","last_became_active":"2024-04-26T19:50:30.113720+0000","last_became_peered":"2024-04-26T19:50:30.113720+0000","last_unstale":"2024-04-26T19:50:38.654064+0000","last_undegraded":"2024-04-26T19:50:38.654064+0000","last_fullsized":"2024-04-26T19:50:38.654064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130140+0000","last_change":"2024-04-26T19:50:35.144698+0000","last_active":"2024-04-26T19:50:37.130140+0000","last_peered":"2024-04-26T19:50:37.130140+0000","last_clean":"2024-04-26T19:50:37.130140+0000","last_became_active":"2024-04-26T19:50:35.144169+0000","last_became_peered":"2024-04-26T19:50:35.144169+0000","last_unstale":"2024-04-26T19:50:37.130140+0000","last_undegraded":"2024-04-26T19:50:37.130140+0000","last_fullsized":"2024-04-26T19:50:37.130140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300094+0000","last_change":"2024-04-26T19:50:31.105626+0000","last_active":"2024-04-26T19:50:38.300094+0000","last_peered":"2024-04-26T19:50:38.300094+0000","last_clean":"2024-04-26T19:50:38.300094+0000","last_became_active":"2024-04-26T19:50:31.105369+0000","last_became_peered":"2024-04-26T19:50:31.105369+0000","last_unstale":"2024-04-26T19:50:38.300094+0000","last_undegraded":"2024-04-26T19:50:38.300094+0000","last_fullsized":"2024-04-26T19:50:38.300094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654017+0000","last_change":"2024-04-26T19:50:30.104092+0000","last_active":"2024-04-26T19:50:38.654017+0000","last_peered":"2024-04-26T19:50:38.654017+0000","last_clean":"2024-04-26T19:50:38.654017+0000","last_became_active":"2024-04-26T19:50:30.103848+0000","last_became_peered":"2024-04-26T19:50:30.103848+0000","last_unstale":"2024-04-26T19:50:38.654017+0000","last_undegraded":"2024-04-26T19:50:38.654017+0000","last_fullsized":"2024-04-26T19:50:38.654017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129689+0000","last_change":"2024-04-26T19:50:35.624499+0000","last_active":"2024-04-26T19:50:37.129689+0000","last_peered":"2024-04-26T19:50:37.129689+0000","last_clean":"2024-04-26T19:50:37.129689+0000","last_became_active":"2024-04-26T19:50:35.624026+0000","last_became_peered":"2024-04-26T19:50:35.624026+0000","last_unstale":"2024-04-26T19:50:37.129689+0000","last_undegraded":"2024-04-26T19:50:37.129689+0000","last_fullsized":"2024-04-26T19:50:37.129689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:46.591736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131049+0000","last_change":"2024-04-26T19:50:31.116347+0000","last_active":"2024-04-26T19:50:37.131049+0000","last_peered":"2024-04-26T19:50:37.131049+0000","last_clean":"2024-04-26T19:50:37.131049+0000","last_became_active":"2024-04-26T19:50:31.116037+0000","last_became_peered":"2024-04-26T19:50:31.116037+0000","last_unstale":"2024-04-26T19:50:37.131049+0000","last_undegraded":"2024-04-26T19:50:37.131049+0000","last_fullsized":"2024-04-26T19:50:37.131049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997937+0000","last_change":"2024-04-26T19:50:30.105663+0000","last_active":"2024-04-26T19:50:37.997937+0000","last_peered":"2024-04-26T19:50:37.997937+0000","last_clean":"2024-04-26T19:50:37.997937+0000","last_became_active":"2024-04-26T19:50:30.105387+0000","last_became_peered":"2024-04-26T19:50:30.105387+0000","last_unstale":"2024-04-26T19:50:37.997937+0000","last_undegraded":"2024-04-26T19:50:37.997937+0000","last_fullsized":"2024-04-26T19:50:37.997937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130198+0000","last_change":"2024-04-26T19:50:35.625444+0000","last_active":"2024-04-26T19:50:37.130198+0000","last_peered":"2024-04-26T19:50:37.130198+0000","last_clean":"2024-04-26T19:50:37.130198+0000","last_became_active":"2024-04-26T19:50:35.625054+0000","last_became_peered":"2024-04-26T19:50:35.625054+0000","last_unstale":"2024-04-26T19:50:37.130198+0000","last_undegraded":"2024-04-26T19:50:37.130198+0000","last_fullsized":"2024-04-26T19:50:37.130198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:07.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997757+0000","last_change":"2024-04-26T19:50:31.109145+0000","last_active":"2024-04-26T19:50:37.997757+0000","last_peered":"2024-04-26T19:50:37.997757+0000","last_clean":"2024-04-26T19:50:37.997757+0000","last_became_active":"2024-04-26T19:50:31.108812+0000","last_became_peered":"2024-04-26T19:50:31.108812+0000","last_unstale":"2024-04-26T19:50:37.997757+0000","last_undegraded":"2024-04-26T19:50:37.997757+0000","last_fullsized":"2024-04-26T19:50:37.997757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653987+0000","last_change":"2024-04-26T19:50:30.112025+0000","last_active":"2024-04-26T19:50:38.653987+0000","last_peered":"2024-04-26T19:50:38.653987+0000","last_clean":"2024-04-26T19:50:38.653987+0000","last_became_active":"2024-04-26T19:50:30.111777+0000","last_became_peered":"2024-04-26T19:50:30.111777+0000","last_unstale":"2024-04-26T19:50:38.653987+0000","last_undegraded":"2024-04-26T19:50:38.653987+0000","last_fullsized":"2024-04-26T19:50:38.653987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:41.838039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299918+0000","last_change":"2024-04-26T19:50:35.623970+0000","last_active":"2024-04-26T19:50:38.299918+0000","last_peered":"2024-04-26T19:50:38.299918+0000","last_clean":"2024-04-26T19:50:38.299918+0000","last_became_active":"2024-04-26T19:50:35.623472+0000","last_became_peered":"2024-04-26T19:50:35.623472+0000","last_unstale":"2024-04-26T19:50:38.299918+0000","last_undegraded":"2024-04-26T19:50:38.299918+0000","last_fullsized":"2024-04-26T19:50:38.299918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654678+0000","last_change":"2024-04-26T19:50:31.120338+0000","last_active":"2024-04-26T19:50:38.654678+0000","last_peered":"2024-04-26T19:50:38.654678+0000","last_clean":"2024-04-26T19:50:38.654678+0000","last_became_active":"2024-04-26T19:50:31.120074+0000","last_became_peered":"2024-04-26T19:50:31.120074+0000","last_unstale":"2024-04-26T19:50:38.654678+0000","last_undegraded":"2024-04-26T19:50:38.654678+0000","last_fullsized":"2024-04-26T19:50:38.654678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997828+0000","last_change":"2024-04-26T19:50:30.106832+0000","last_active":"2024-04-26T19:50:37.997828+0000","last_peered":"2024-04-26T19:50:37.997828+0000","last_clean":"2024-04-26T19:50:37.997828+0000","last_became_active":"2024-04-26T19:50:30.106553+0000","last_became_peered":"2024-04-26T19:50:30.106553+0000","last_unstale":"2024-04-26T19:50:37.997828+0000","last_undegraded":"2024-04-26T19:50:37.997828+0000","last_fullsized":"2024-04-26T19:50:37.997828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997421+0000","last_change":"2024-04-26T19:50:35.149679+0000","last_active":"2024-04-26T19:50:37.997421+0000","last_peered":"2024-04-26T19:50:37.997421+0000","last_clean":"2024-04-26T19:50:37.997421+0000","last_became_active":"2024-04-26T19:50:35.149304+0000","last_became_peered":"2024-04-26T19:50:35.149304+0000","last_unstale":"2024-04-26T19:50:37.997421+0000","last_undegraded":"2024-04-26T19:50:37.997421+0000","last_fullsized":"2024-04-26T19:50:37.997421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130385+0000","last_change":"2024-04-26T19:50:31.105502+0000","last_active":"2024-04-26T19:50:37.130385+0000","last_peered":"2024-04-26T19:50:37.130385+0000","last_clean":"2024-04-26T19:50:37.130385+0000","last_became_active":"2024-04-26T19:50:31.105047+0000","last_became_peered":"2024-04-26T19:50:31.105047+0000","last_unstale":"2024-04-26T19:50:37.130385+0000","last_undegraded":"2024-04-26T19:50:37.130385+0000","last_fullsized":"2024-04-26T19:50:37.130385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:20.874838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653576+0000","last_change":"2024-04-26T19:50:30.110096+0000","last_active":"2024-04-26T19:50:38.653576+0000","last_peered":"2024-04-26T19:50:38.653576+0000","last_clean":"2024-04-26T19:50:38.653576+0000","last_became_active":"2024-04-26T19:50:30.109131+0000","last_became_peered":"2024-04-26T19:50:30.109131+0000","last_unstale":"2024-04-26T19:50:38.653576+0000","last_undegraded":"2024-04-26T19:50:38.653576+0000","last_fullsized":"2024-04-26T19:50:38.653576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997740+0000","last_change":"2024-04-26T19:50:35.148072+0000","last_active":"2024-04-26T19:50:37.997740+0000","last_peered":"2024-04-26T19:50:37.997740+0000","last_clean":"2024-04-26T19:50:37.997740+0000","last_became_active":"2024-04-26T19:50:35.147697+0000","last_became_peered":"2024-04-26T19:50:35.147697+0000","last_unstale":"2024-04-26T19:50:37.997740+0000","last_undegraded":"2024-04-26T19:50:37.997740+0000","last_fullsized":"2024-04-26T19:50:37.997740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997366+0000","last_change":"2024-04-26T19:50:31.108339+0000","last_active":"2024-04-26T19:50:37.997366+0000","last_peered":"2024-04-26T19:50:37.997366+0000","last_clean":"2024-04-26T19:50:37.997366+0000","last_became_active":"2024-04-26T19:50:31.108056+0000","last_became_peered":"2024-04-26T19:50:31.108056+0000","last_unstale":"2024-04-26T19:50:37.997366+0000","last_undegraded":"2024-04-26T19:50:37.997366+0000","last_fullsized":"2024-04-26T19:50:37.997366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536841+0000","last_change":"2024-04-26T19:50:30.111350+0000","last_active":"2024-04-26T19:50:38.536841+0000","last_peered":"2024-04-26T19:50:38.536841+0000","last_clean":"2024-04-26T19:50:38.536841+0000","last_became_active":"2024-04-26T19:50:30.111070+0000","last_became_peered":"2024-04-26T19:50:30.111070+0000","last_unstale":"2024-04-26T19:50:38.536841+0000","last_undegraded":"2024-04-26T19:50:38.536841+0000","last_fullsized":"2024-04-26T19:50:38.536841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.998002+0000","last_change":"2024-04-26T19:50:35.146694+0000","last_active":"2024-04-26T19:50:37.998002+0000","last_peered":"2024-04-26T19:50:37.998002+0000","last_clean":"2024-04-26T19:50:37.998002+0000","last_became_active":"2024-04-26T19:50:35.145805+0000","last_became_peered":"2024-04-26T19:50:35.145805+0000","last_unstale":"2024-04-26T19:50:37.998002+0000","last_undegraded":"2024-04-26T19:50:37.998002+0000","last_fullsized":"2024-04-26T19:50:37.998002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:01.350601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299548+0000","last_change":"2024-04-26T19:50:31.103279+0000","last_active":"2024-04-26T19:50:38.299548+0000","last_peered":"2024-04-26T19:50:38.299548+0000","last_clean":"2024-04-26T19:50:38.299548+0000","last_became_active":"2024-04-26T19:50:31.102858+0000","last_became_peered":"2024-04-26T19:50:31.102858+0000","last_unstale":"2024-04-26T19:50:38.299548+0000","last_undegraded":"2024-04-26T19:50:38.299548+0000","last_fullsized":"2024-04-26T19:50:38.299548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:56.497144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653771+0000","last_change":"2024-04-26T19:50:30.109372+0000","last_active":"2024-04-26T19:50:38.653771+0000","last_peered":"2024-04-26T19:50:38.653771+0000","last_clean":"2024-04-26T19:50:38.653771+0000","last_became_active":"2024-04-26T19:50:30.109087+0000","last_became_peered":"2024-04-26T19:50:30.109087+0000","last_unstale":"2024-04-26T19:50:38.653771+0000","last_undegraded":"2024-04-26T19:50:38.653771+0000","last_fullsized":"2024-04-26T19:50:38.653771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:13.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300021+0000","last_change":"2024-04-26T19:50:35.623867+0000","last_active":"2024-04-26T19:50:38.300021+0000","last_peered":"2024-04-26T19:50:38.300021+0000","last_clean":"2024-04-26T19:50:38.300021+0000","last_became_active":"2024-04-26T19:50:35.623416+0000","last_became_peered":"2024-04-26T19:50:35.623416+0000","last_unstale":"2024-04-26T19:50:38.300021+0000","last_undegraded":"2024-04-26T19:50:38.300021+0000","last_fullsized":"2024-04-26T19:50:38.300021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:02.874670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130394+0000","last_change":"2024-04-26T19:50:31.113014+0000","last_active":"2024-04-26T19:50:37.130394+0000","last_peered":"2024-04-26T19:50:37.130394+0000","last_clean":"2024-04-26T19:50:37.130394+0000","last_became_active":"2024-04-26T19:50:31.112753+0000","last_became_peered":"2024-04-26T19:50:31.112753+0000","last_unstale":"2024-04-26T19:50:37.130394+0000","last_undegraded":"2024-04-26T19:50:37.130394+0000","last_fullsized":"2024-04-26T19:50:37.130394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653084+0000","last_change":"2024-04-26T19:50:30.110312+0000","last_active":"2024-04-26T19:50:38.653084+0000","last_peered":"2024-04-26T19:50:38.653084+0000","last_clean":"2024-04-26T19:50:38.653084+0000","last_became_active":"2024-04-26T19:50:30.109147+0000","last_became_peered":"2024-04-26T19:50:30.109147+0000","last_unstale":"2024-04-26T19:50:38.653084+0000","last_undegraded":"2024-04-26T19:50:38.653084+0000","last_fullsized":"2024-04-26T19:50:38.653084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654171+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:50:38.654171+0000","last_peered":"2024-04-26T19:50:38.654171+0000","last_clean":"2024-04-26T19:50:38.654171+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:50:38.654171+0000","last_undegraded":"2024-04-26T19:50:38.654171+0000","last_fullsized":"2024-04-26T19:50:38.654171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653046+0000","last_change":"2024-04-26T19:50:35.627345+0000","last_active":"2024-04-26T19:50:38.653046+0000","last_peered":"2024-04-26T19:50:38.653046+0000","last_clean":"2024-04-26T19:50:38.653046+0000","last_became_active":"2024-04-26T19:50:35.626795+0000","last_became_peered":"2024-04-26T19:50:35.626795+0000","last_unstale":"2024-04-26T19:50:38.653046+0000","last_undegraded":"2024-04-26T19:50:38.653046+0000","last_fullsized":"2024-04-26T19:50:38.653046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:14.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653270+0000","last_change":"2024-04-26T19:50:31.112950+0000","last_active":"2024-04-26T19:50:38.653270+0000","last_peered":"2024-04-26T19:50:38.653270+0000","last_clean":"2024-04-26T19:50:38.653270+0000","last_became_active":"2024-04-26T19:50:31.112674+0000","last_became_peered":"2024-04-26T19:50:31.112674+0000","last_unstale":"2024-04-26T19:50:38.653270+0000","last_undegraded":"2024-04-26T19:50:38.653270+0000","last_fullsized":"2024-04-26T19:50:38.653270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537253+0000","last_change":"2024-04-26T19:50:30.114819+0000","last_active":"2024-04-26T19:50:38.537253+0000","last_peered":"2024-04-26T19:50:38.537253+0000","last_clean":"2024-04-26T19:50:38.537253+0000","last_became_active":"2024-04-26T19:50:30.114566+0000","last_became_peered":"2024-04-26T19:50:30.114566+0000","last_unstale":"2024-04-26T19:50:38.537253+0000","last_undegraded":"2024-04-26T19:50:38.537253+0000","last_fullsized":"2024-04-26T19:50:38.537253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654387+0000","last_change":"2024-04-26T19:50:35.626488+0000","last_active":"2024-04-26T19:50:38.654387+0000","last_peered":"2024-04-26T19:50:38.654387+0000","last_clean":"2024-04-26T19:50:38.654387+0000","last_became_active":"2024-04-26T19:50:35.625951+0000","last_became_peered":"2024-04-26T19:50:35.625951+0000","last_unstale":"2024-04-26T19:50:38.654387+0000","last_undegraded":"2024-04-26T19:50:38.654387+0000","last_fullsized":"2024-04-26T19:50:38.654387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:46.591736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652684+0000","last_change":"2024-04-26T19:50:31.119380+0000","last_active":"2024-04-26T19:50:38.652684+0000","last_peered":"2024-04-26T19:50:38.652684+0000","last_clean":"2024-04-26T19:50:38.652684+0000","last_became_active":"2024-04-26T19:50:31.118011+0000","last_became_peered":"2024-04-26T19:50:31.118011+0000","last_unstale":"2024-04-26T19:50:38.652684+0000","last_undegraded":"2024-04-26T19:50:38.652684+0000","last_fullsized":"2024-04-26T19:50:38.652684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:13.577254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997787+0000","last_change":"2024-04-26T19:50:30.101571+0000","last_active":"2024-04-26T19:50:37.997787+0000","last_peered":"2024-04-26T19:50:37.997787+0000","last_clean":"2024-04-26T19:50:37.997787+0000","last_became_active":"2024-04-26T19:50:30.101257+0000","last_became_peered":"2024-04-26T19:50:30.101257+0000","last_unstale":"2024-04-26T19:50:37.997787+0000","last_undegraded":"2024-04-26T19:50:37.997787+0000","last_fullsized":"2024-04-26T19:50:37.997787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:13.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299968+0000","last_change":"2024-04-26T19:50:35.146304+0000","last_active":"2024-04-26T19:50:38.299968+0000","last_peered":"2024-04-26T19:50:38.299968+0000","last_clean":"2024-04-26T19:50:38.299968+0000","last_became_active":"2024-04-26T19:50:35.145918+0000","last_became_peered":"2024-04-26T19:50:35.145918+0000","last_unstale":"2024-04-26T19:50:38.299968+0000","last_undegraded":"2024-04-26T19:50:38.299968+0000","last_fullsized":"2024-04-26T19:50:38.299968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653338+0000","last_change":"2024-04-26T19:50:31.115622+0000","last_active":"2024-04-26T19:50:38.653338+0000","last_peered":"2024-04-26T19:50:38.653338+0000","last_clean":"2024-04-26T19:50:38.653338+0000","last_became_active":"2024-04-26T19:50:31.115358+0000","last_became_peered":"2024-04-26T19:50:31.115358+0000","last_unstale":"2024-04-26T19:50:38.653338+0000","last_undegraded":"2024-04-26T19:50:38.653338+0000","last_fullsized":"2024-04-26T19:50:38.653338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:31.186700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536869+0000","last_change":"2024-04-26T19:50:30.112087+0000","last_active":"2024-04-26T19:50:38.536869+0000","last_peered":"2024-04-26T19:50:38.536869+0000","last_clean":"2024-04-26T19:50:38.536869+0000","last_became_active":"2024-04-26T19:50:30.111773+0000","last_became_peered":"2024-04-26T19:50:30.111773+0000","last_unstale":"2024-04-26T19:50:38.536869+0000","last_undegraded":"2024-04-26T19:50:38.536869+0000","last_fullsized":"2024-04-26T19:50:38.536869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652160+0000","last_change":"2024-04-26T19:50:35.624170+0000","last_active":"2024-04-26T19:50:38.652160+0000","last_peered":"2024-04-26T19:50:38.652160+0000","last_clean":"2024-04-26T19:50:38.652160+0000","last_became_active":"2024-04-26T19:50:35.623758+0000","last_became_peered":"2024-04-26T19:50:35.623758+0000","last_unstale":"2024-04-26T19:50:38.652160+0000","last_undegraded":"2024-04-26T19:50:38.652160+0000","last_fullsized":"2024-04-26T19:50:38.652160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:01.350601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653614+0000","last_change":"2024-04-26T19:50:31.115100+0000","last_active":"2024-04-26T19:50:38.653614+0000","last_peered":"2024-04-26T19:50:38.653614+0000","last_clean":"2024-04-26T19:50:38.653614+0000","last_became_active":"2024-04-26T19:50:31.114558+0000","last_became_peered":"2024-04-26T19:50:31.114558+0000","last_unstale":"2024-04-26T19:50:38.653614+0000","last_undegraded":"2024-04-26T19:50:38.653614+0000","last_fullsized":"2024-04-26T19:50:38.653614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:42.558104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130407+0000","last_change":"2024-04-26T19:50:30.100580+0000","last_active":"2024-04-26T19:50:37.130407+0000","last_peered":"2024-04-26T19:50:37.130407+0000","last_clean":"2024-04-26T19:50:37.130407+0000","last_became_active":"2024-04-26T19:50:30.099562+0000","last_became_peered":"2024-04-26T19:50:30.099562+0000","last_unstale":"2024-04-26T19:50:37.130407+0000","last_undegraded":"2024-04-26T19:50:37.130407+0000","last_fullsized":"2024-04-26T19:50:37.130407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:02.207190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129529+0000","last_change":"2024-04-26T19:50:35.143446+0000","last_active":"2024-04-26T19:50:37.129529+0000","last_peered":"2024-04-26T19:50:37.129529+0000","last_clean":"2024-04-26T19:50:37.129529+0000","last_became_active":"2024-04-26T19:50:35.143028+0000","last_became_peered":"2024-04-26T19:50:35.143028+0000","last_unstale":"2024-04-26T19:50:37.129529+0000","last_undegraded":"2024-04-26T19:50:37.129529+0000","last_fullsized":"2024-04-26T19:50:37.129529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:43.974896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653088+0000","last_change":"2024-04-26T19:50:31.116546+0000","last_active":"2024-04-26T19:50:38.653088+0000","last_peered":"2024-04-26T19:50:38.653088+0000","last_clean":"2024-04-26T19:50:38.653088+0000","last_became_active":"2024-04-26T19:50:31.115261+0000","last_became_peered":"2024-04-26T19:50:31.115261+0000","last_unstale":"2024-04-26T19:50:38.653088+0000","last_undegraded":"2024-04-26T19:50:38.653088+0000","last_fullsized":"2024-04-26T19:50:38.653088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.212877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536837+0000","last_change":"2024-04-26T19:50:30.099768+0000","last_active":"2024-04-26T19:50:38.536837+0000","last_peered":"2024-04-26T19:50:38.536837+0000","last_clean":"2024-04-26T19:50:38.536837+0000","last_became_active":"2024-04-26T19:50:30.099471+0000","last_became_peered":"2024-04-26T19:50:30.099471+0000","last_unstale":"2024-04-26T19:50:38.536837+0000","last_undegraded":"2024-04-26T19:50:38.536837+0000","last_fullsized":"2024-04-26T19:50:38.536837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299923+0000","last_change":"2024-04-26T19:50:35.148077+0000","last_active":"2024-04-26T19:50:38.299923+0000","last_peered":"2024-04-26T19:50:38.299923+0000","last_clean":"2024-04-26T19:50:38.299923+0000","last_became_active":"2024-04-26T19:50:35.147552+0000","last_became_peered":"2024-04-26T19:50:35.147552+0000","last_unstale":"2024-04-26T19:50:38.299923+0000","last_undegraded":"2024-04-26T19:50:38.299923+0000","last_fullsized":"2024-04-26T19:50:38.299923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:11.052818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652600+0000","last_change":"2024-04-26T19:50:31.105398+0000","last_active":"2024-04-26T19:50:38.652600+0000","last_peered":"2024-04-26T19:50:38.652600+0000","last_clean":"2024-04-26T19:50:38.652600+0000","last_became_active":"2024-04-26T19:50:31.105140+0000","last_became_peered":"2024-04-26T19:50:31.105140+0000","last_unstale":"2024-04-26T19:50:38.652600+0000","last_undegraded":"2024-04-26T19:50:38.652600+0000","last_fullsized":"2024-04-26T19:50:38.652600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.651966+0000","last_change":"2024-04-26T19:50:30.111019+0000","last_active":"2024-04-26T19:50:38.651966+0000","last_peered":"2024-04-26T19:50:38.651966+0000","last_clean":"2024-04-26T19:50:38.651966+0000","last_became_active":"2024-04-26T19:50:30.110590+0000","last_became_peered":"2024-04-26T19:50:30.110590+0000","last_unstale":"2024-04-26T19:50:38.651966+0000","last_undegraded":"2024-04-26T19:50:38.651966+0000","last_fullsized":"2024-04-26T19:50:38.651966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131001+0000","last_change":"2024-04-26T19:50:35.625721+0000","last_active":"2024-04-26T19:50:37.131001+0000","last_peered":"2024-04-26T19:50:37.131001+0000","last_clean":"2024-04-26T19:50:37.131001+0000","last_became_active":"2024-04-26T19:50:35.625294+0000","last_became_peered":"2024-04-26T19:50:35.625294+0000","last_unstale":"2024-04-26T19:50:37.131001+0000","last_undegraded":"2024-04-26T19:50:37.131001+0000","last_fullsized":"2024-04-26T19:50:37.131001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:18.158960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653765+0000","last_change":"2024-04-26T19:50:31.113984+0000","last_active":"2024-04-26T19:50:38.653765+0000","last_peered":"2024-04-26T19:50:38.653765+0000","last_clean":"2024-04-26T19:50:38.653765+0000","last_became_active":"2024-04-26T19:50:31.113723+0000","last_became_peered":"2024-04-26T19:50:31.113723+0000","last_unstale":"2024-04-26T19:50:38.653765+0000","last_undegraded":"2024-04-26T19:50:38.653765+0000","last_fullsized":"2024-04-26T19:50:38.653765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653698+0000","last_change":"2024-04-26T19:50:30.110687+0000","last_active":"2024-04-26T19:50:38.653698+0000","last_peered":"2024-04-26T19:50:38.653698+0000","last_clean":"2024-04-26T19:50:38.653698+0000","last_became_active":"2024-04-26T19:50:30.110423+0000","last_became_peered":"2024-04-26T19:50:30.110423+0000","last_unstale":"2024-04-26T19:50:38.653698+0000","last_undegraded":"2024-04-26T19:50:38.653698+0000","last_fullsized":"2024-04-26T19:50:38.653698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130741+0000","last_change":"2024-04-26T19:50:35.623855+0000","last_active":"2024-04-26T19:50:37.130741+0000","last_peered":"2024-04-26T19:50:37.130741+0000","last_clean":"2024-04-26T19:50:37.130741+0000","last_became_active":"2024-04-26T19:50:35.623453+0000","last_became_peered":"2024-04-26T19:50:35.623453+0000","last_unstale":"2024-04-26T19:50:37.130741+0000","last_undegraded":"2024-04-26T19:50:37.130741+0000","last_fullsized":"2024-04-26T19:50:37.130741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:21.199810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299726+0000","last_change":"2024-04-26T19:50:31.114891+0000","last_active":"2024-04-26T19:50:38.299726+0000","last_peered":"2024-04-26T19:50:38.299726+0000","last_clean":"2024-04-26T19:50:38.299726+0000","last_became_active":"2024-04-26T19:50:31.114338+0000","last_became_peered":"2024-04-26T19:50:31.114338+0000","last_unstale":"2024-04-26T19:50:38.299726+0000","last_undegraded":"2024-04-26T19:50:38.299726+0000","last_fullsized":"2024-04-26T19:50:38.299726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299301+0000","last_change":"2024-04-26T19:50:30.108408+0000","last_active":"2024-04-26T19:50:38.299301+0000","last_peered":"2024-04-26T19:50:38.299301+0000","last_clean":"2024-04-26T19:50:38.299301+0000","last_became_active":"2024-04-26T19:50:30.107261+0000","last_became_peered":"2024-04-26T19:50:30.107261+0000","last_unstale":"2024-04-26T19:50:38.299301+0000","last_undegraded":"2024-04-26T19:50:38.299301+0000","last_fullsized":"2024-04-26T19:50:38.299301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:21.153340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654534+0000","last_change":"2024-04-26T19:50:35.624403+0000","last_active":"2024-04-26T19:50:38.654534+0000","last_peered":"2024-04-26T19:50:38.654534+0000","last_clean":"2024-04-26T19:50:38.654534+0000","last_became_active":"2024-04-26T19:50:35.623811+0000","last_became_peered":"2024-04-26T19:50:35.623811+0000","last_unstale":"2024-04-26T19:50:38.654534+0000","last_undegraded":"2024-04-26T19:50:38.654534+0000","last_fullsized":"2024-04-26T19:50:38.654534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.747785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653171+0000","last_change":"2024-04-26T19:50:31.115452+0000","last_active":"2024-04-26T19:50:38.653171+0000","last_peered":"2024-04-26T19:50:38.653171+0000","last_clean":"2024-04-26T19:50:38.653171+0000","last_became_active":"2024-04-26T19:50:31.114780+0000","last_became_peered":"2024-04-26T19:50:31.114780+0000","last_unstale":"2024-04-26T19:50:38.653171+0000","last_undegraded":"2024-04-26T19:50:38.653171+0000","last_fullsized":"2024-04-26T19:50:38.653171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:40.502106+0000","last_change":"2024-04-26T19:50:30.110540+0000","last_active":"2024-04-26T19:50:40.502106+0000","last_peered":"2024-04-26T19:50:40.502106+0000","last_clean":"2024-04-26T19:50:40.502106+0000","last_became_active":"2024-04-26T19:50:30.110241+0000","last_became_peered":"2024-04-26T19:50:30.110241+0000","last_unstale":"2024-04-26T19:50:40.502106+0000","last_undegraded":"2024-04-26T19:50:40.502106+0000","last_fullsized":"2024-04-26T19:50:40.502106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997615+0000","last_change":"2024-04-26T19:50:35.149139+0000","last_active":"2024-04-26T19:50:37.997615+0000","last_peered":"2024-04-26T19:50:37.997615+0000","last_clean":"2024-04-26T19:50:37.997615+0000","last_became_active":"2024-04-26T19:50:35.148663+0000","last_became_peered":"2024-04-26T19:50:35.148663+0000","last_unstale":"2024-04-26T19:50:37.997615+0000","last_undegraded":"2024-04-26T19:50:37.997615+0000","last_fullsized":"2024-04-26T19:50:37.997615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130224+0000","last_change":"2024-04-26T19:50:31.112921+0000","last_active":"2024-04-26T19:50:37.130224+0000","last_peered":"2024-04-26T19:50:37.130224+0000","last_clean":"2024-04-26T19:50:37.130224+0000","last_became_active":"2024-04-26T19:50:31.112651+0000","last_became_peered":"2024-04-26T19:50:31.112651+0000","last_unstale":"2024-04-26T19:50:37.130224+0000","last_undegraded":"2024-04-26T19:50:37.130224+0000","last_fullsized":"2024-04-26T19:50:37.130224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:13.149763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997777+0000","last_change":"2024-04-26T19:50:30.110464+0000","last_active":"2024-04-26T19:50:37.997777+0000","last_peered":"2024-04-26T19:50:37.997777+0000","last_clean":"2024-04-26T19:50:37.997777+0000","last_became_active":"2024-04-26T19:50:30.110101+0000","last_became_peered":"2024-04-26T19:50:30.110101+0000","last_unstale":"2024-04-26T19:50:37.997777+0000","last_undegraded":"2024-04-26T19:50:37.997777+0000","last_fullsized":"2024-04-26T19:50:37.997777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:12.144175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130067+0000","last_change":"2024-04-26T19:50:35.143854+0000","last_active":"2024-04-26T19:50:37.130067+0000","last_peered":"2024-04-26T19:50:37.130067+0000","last_clean":"2024-04-26T19:50:37.130067+0000","last_became_active":"2024-04-26T19:50:35.143264+0000","last_became_peered":"2024-04-26T19:50:35.143264+0000","last_unstale":"2024-04-26T19:50:37.130067+0000","last_undegraded":"2024-04-26T19:50:37.130067+0000","last_fullsized":"2024-04-26T19:50:37.130067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300209+0000","last_change":"2024-04-26T19:50:31.115638+0000","last_active":"2024-04-26T19:50:38.300209+0000","last_peered":"2024-04-26T19:50:38.300209+0000","last_clean":"2024-04-26T19:50:38.300209+0000","last_became_active":"2024-04-26T19:50:31.113889+0000","last_became_peered":"2024-04-26T19:50:31.113889+0000","last_unstale":"2024-04-26T19:50:38.300209+0000","last_undegraded":"2024-04-26T19:50:38.300209+0000","last_fullsized":"2024-04-26T19:50:38.300209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652958+0000","last_change":"2024-04-26T19:50:30.109848+0000","last_active":"2024-04-26T19:50:38.652958+0000","last_peered":"2024-04-26T19:50:38.652958+0000","last_clean":"2024-04-26T19:50:38.652958+0000","last_became_active":"2024-04-26T19:50:30.109407+0000","last_became_peered":"2024-04-26T19:50:30.109407+0000","last_unstale":"2024-04-26T19:50:38.652958+0000","last_undegraded":"2024-04-26T19:50:38.652958+0000","last_fullsized":"2024-04-26T19:50:38.652958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130343+0000","last_change":"2024-04-26T19:50:35.143221+0000","last_active":"2024-04-26T19:50:37.130343+0000","last_peered":"2024-04-26T19:50:37.130343+0000","last_clean":"2024-04-26T19:50:37.130343+0000","last_became_active":"2024-04-26T19:50:35.142868+0000","last_became_peered":"2024-04-26T19:50:35.142868+0000","last_unstale":"2024-04-26T19:50:37.130343+0000","last_undegraded":"2024-04-26T19:50:37.130343+0000","last_fullsized":"2024-04-26T19:50:37.130343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:16.097115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652135+0000","last_change":"2024-04-26T19:50:31.115770+0000","last_active":"2024-04-26T19:50:38.652135+0000","last_peered":"2024-04-26T19:50:38.652135+0000","last_clean":"2024-04-26T19:50:38.652135+0000","last_became_active":"2024-04-26T19:50:31.115462+0000","last_became_peered":"2024-04-26T19:50:31.115462+0000","last_unstale":"2024-04-26T19:50:38.652135+0000","last_undegraded":"2024-04-26T19:50:38.652135+0000","last_fullsized":"2024-04-26T19:50:38.652135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.212877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653880+0000","last_change":"2024-04-26T19:50:30.110213+0000","last_active":"2024-04-26T19:50:38.653880+0000","last_peered":"2024-04-26T19:50:38.653880+0000","last_clean":"2024-04-26T19:50:38.653880+0000","last_became_active":"2024-04-26T19:50:30.109923+0000","last_became_peered":"2024-04-26T19:50:30.109923+0000","last_unstale":"2024-04-26T19:50:38.653880+0000","last_undegraded":"2024-04-26T19:50:38.653880+0000","last_fullsized":"2024-04-26T19:50:38.653880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130324+0000","last_change":"2024-04-26T19:50:35.624637+0000","last_active":"2024-04-26T19:50:37.130324+0000","last_peered":"2024-04-26T19:50:37.130324+0000","last_clean":"2024-04-26T19:50:37.130324+0000","last_became_active":"2024-04-26T19:50:35.624195+0000","last_became_peered":"2024-04-26T19:50:35.624195+0000","last_unstale":"2024-04-26T19:50:37.130324+0000","last_undegraded":"2024-04-26T19:50:37.130324+0000","last_fullsized":"2024-04-26T19:50:37.130324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:16.049053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299925+0000","last_change":"2024-04-26T19:50:35.626111+0000","last_active":"2024-04-26T19:50:38.299925+0000","last_peered":"2024-04-26T19:50:38.299925+0000","last_clean":"2024-04-26T19:50:38.299925+0000","last_became_active":"2024-04-26T19:50:35.625678+0000","last_became_peered":"2024-04-26T19:50:35.625678+0000","last_unstale":"2024-04-26T19:50:38.299925+0000","last_undegraded":"2024-04-26T19:50:38.299925+0000","last_fullsized":"2024-04-26T19:50:38.299925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997291+0000","last_change":"2024-04-26T19:50:31.108163+0000","last_active":"2024-04-26T19:50:37.997291+0000","last_peered":"2024-04-26T19:50:37.997291+0000","last_clean":"2024-04-26T19:50:37.997291+0000","last_became_active":"2024-04-26T19:50:31.107840+0000","last_became_peered":"2024-04-26T19:50:31.107840+0000","last_unstale":"2024-04-26T19:50:37.997291+0000","last_undegraded":"2024-04-26T19:50:37.997291+0000","last_fullsized":"2024-04-26T19:50:37.997291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.651925+0000","last_change":"2024-04-26T19:50:30.111205+0000","last_active":"2024-04-26T19:50:38.651925+0000","last_peered":"2024-04-26T19:50:38.651925+0000","last_clean":"2024-04-26T19:50:38.651925+0000","last_became_active":"2024-04-26T19:50:30.110491+0000","last_became_peered":"2024-04-26T19:50:30.110491+0000","last_unstale":"2024-04-26T19:50:38.651925+0000","last_undegraded":"2024-04-26T19:50:38.651925+0000","last_fullsized":"2024-04-26T19:50:38.651925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653880+0000","last_change":"2024-04-26T19:50:35.144010+0000","last_active":"2024-04-26T19:50:38.653880+0000","last_peered":"2024-04-26T19:50:38.653880+0000","last_clean":"2024-04-26T19:50:38.653880+0000","last_became_active":"2024-04-26T19:50:35.143558+0000","last_became_peered":"2024-04-26T19:50:35.143558+0000","last_unstale":"2024-04-26T19:50:38.653880+0000","last_undegraded":"2024-04-26T19:50:38.653880+0000","last_fullsized":"2024-04-26T19:50:38.653880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:10.325286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130338+0000","last_change":"2024-04-26T19:50:31.106592+0000","last_active":"2024-04-26T19:50:37.130338+0000","last_peered":"2024-04-26T19:50:37.130338+0000","last_clean":"2024-04-26T19:50:37.130338+0000","last_became_active":"2024-04-26T19:50:31.106298+0000","last_became_peered":"2024-04-26T19:50:31.106298+0000","last_unstale":"2024-04-26T19:50:37.130338+0000","last_undegraded":"2024-04-26T19:50:37.130338+0000","last_fullsized":"2024-04-26T19:50:37.130338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130435+0000","last_change":"2024-04-26T19:50:30.110713+0000","last_active":"2024-04-26T19:50:37.130435+0000","last_peered":"2024-04-26T19:50:37.130435+0000","last_clean":"2024-04-26T19:50:37.130435+0000","last_became_active":"2024-04-26T19:50:30.109274+0000","last_became_peered":"2024-04-26T19:50:30.109274+0000","last_unstale":"2024-04-26T19:50:37.130435+0000","last_undegraded":"2024-04-26T19:50:37.130435+0000","last_fullsized":"2024-04-26T19:50:37.130435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654164+0000","last_change":"2024-04-26T19:50:30.111443+0000","last_active":"2024-04-26T19:50:38.654164+0000","last_peered":"2024-04-26T19:50:38.654164+0000","last_clean":"2024-04-26T19:50:38.654164+0000","last_became_active":"2024-04-26T19:50:30.111159+0000","last_became_peered":"2024-04-26T19:50:30.111159+0000","last_unstale":"2024-04-26T19:50:38.654164+0000","last_undegraded":"2024-04-26T19:50:38.654164+0000","last_fullsized":"2024-04-26T19:50:38.654164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300079+0000","last_change":"2024-04-26T19:50:31.104770+0000","last_active":"2024-04-26T19:50:38.300079+0000","last_peered":"2024-04-26T19:50:38.300079+0000","last_clean":"2024-04-26T19:50:38.300079+0000","last_became_active":"2024-04-26T19:50:31.104100+0000","last_became_peered":"2024-04-26T19:50:31.104100+0000","last_unstale":"2024-04-26T19:50:38.300079+0000","last_undegraded":"2024-04-26T19:50:38.300079+0000","last_fullsized":"2024-04-26T19:50:38.300079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537176+0000","last_change":"2024-04-26T19:50:35.143179+0000","last_active":"2024-04-26T19:50:38.537176+0000","last_peered":"2024-04-26T19:50:38.537176+0000","last_clean":"2024-04-26T19:50:38.537176+0000","last_became_active":"2024-04-26T19:50:35.142761+0000","last_became_peered":"2024-04-26T19:50:35.142761+0000","last_unstale":"2024-04-26T19:50:38.537176+0000","last_undegraded":"2024-04-26T19:50:38.537176+0000","last_fullsized":"2024-04-26T19:50:38.537176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:42.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130613+0000","last_change":"2024-04-26T19:50:30.115508+0000","last_active":"2024-04-26T19:50:37.130613+0000","last_peered":"2024-04-26T19:50:37.130613+0000","last_clean":"2024-04-26T19:50:37.130613+0000","last_became_active":"2024-04-26T19:50:30.115235+0000","last_became_peered":"2024-04-26T19:50:30.115235+0000","last_unstale":"2024-04-26T19:50:37.130613+0000","last_undegraded":"2024-04-26T19:50:37.130613+0000","last_fullsized":"2024-04-26T19:50:37.130613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:31.691383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129914+0000","last_change":"2024-04-26T19:50:31.110879+0000","last_active":"2024-04-26T19:50:37.129914+0000","last_peered":"2024-04-26T19:50:37.129914+0000","last_clean":"2024-04-26T19:50:37.129914+0000","last_became_active":"2024-04-26T19:50:31.110592+0000","last_became_peered":"2024-04-26T19:50:31.110592+0000","last_unstale":"2024-04-26T19:50:37.129914+0000","last_undegraded":"2024-04-26T19:50:37.129914+0000","last_fullsized":"2024-04-26T19:50:37.129914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:54.410618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130887+0000","last_change":"2024-04-26T19:50:35.144257+0000","last_active":"2024-04-26T19:50:37.130887+0000","last_peered":"2024-04-26T19:50:37.130887+0000","last_clean":"2024-04-26T19:50:37.130887+0000","last_became_active":"2024-04-26T19:50:35.143853+0000","last_became_peered":"2024-04-26T19:50:35.143853+0000","last_unstale":"2024-04-26T19:50:37.130887+0000","last_undegraded":"2024-04-26T19:50:37.130887+0000","last_fullsized":"2024-04-26T19:50:37.130887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:05.350339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652235+0000","last_change":"2024-04-26T19:50:31.105355+0000","last_active":"2024-04-26T19:50:38.652235+0000","last_peered":"2024-04-26T19:50:38.652235+0000","last_clean":"2024-04-26T19:50:38.652235+0000","last_became_active":"2024-04-26T19:50:31.105085+0000","last_became_peered":"2024-04-26T19:50:31.105085+0000","last_unstale":"2024-04-26T19:50:38.652235+0000","last_undegraded":"2024-04-26T19:50:38.652235+0000","last_fullsized":"2024-04-26T19:50:38.652235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536962+0000","last_change":"2024-04-26T19:50:31.112415+0000","last_active":"2024-04-26T19:50:38.536962+0000","last_peered":"2024-04-26T19:50:38.536962+0000","last_clean":"2024-04-26T19:50:38.536962+0000","last_became_active":"2024-04-26T19:50:31.112126+0000","last_became_peered":"2024-04-26T19:50:31.112126+0000","last_unstale":"2024-04-26T19:50:38.536962+0000","last_undegraded":"2024-04-26T19:50:38.536962+0000","last_fullsized":"2024-04-26T19:50:38.536962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:13.149763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654244+0000","last_change":"2024-04-26T19:50:31.121580+0000","last_active":"2024-04-26T19:50:38.654244+0000","last_peered":"2024-04-26T19:50:38.654244+0000","last_clean":"2024-04-26T19:50:38.654244+0000","last_became_active":"2024-04-26T19:50:31.121296+0000","last_became_peered":"2024-04-26T19:50:31.121296+0000","last_unstale":"2024-04-26T19:50:38.654244+0000","last_undegraded":"2024-04-26T19:50:38.654244+0000","last_fullsized":"2024-04-26T19:50:38.654244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.291653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300008+0000","last_change":"2024-04-26T19:50:31.106600+0000","last_active":"2024-04-26T19:50:38.300008+0000","last_peered":"2024-04-26T19:50:38.300008+0000","last_clean":"2024-04-26T19:50:38.300008+0000","last_became_active":"2024-04-26T19:50:31.106120+0000","last_became_peered":"2024-04-26T19:50:31.106120+0000","last_unstale":"2024-04-26T19:50:38.300008+0000","last_undegraded":"2024-04-26T19:50:38.300008+0000","last_fullsized":"2024-04-26T19:50:38.300008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130813+0000","last_change":"2024-04-26T19:50:31.113865+0000","last_active":"2024-04-26T19:50:37.130813+0000","last_peered":"2024-04-26T19:50:37.130813+0000","last_clean":"2024-04-26T19:50:37.130813+0000","last_became_active":"2024-04-26T19:50:31.113603+0000","last_became_peered":"2024-04-26T19:50:31.113603+0000","last_unstale":"2024-04-26T19:50:37.130813+0000","last_undegraded":"2024-04-26T19:50:37.130813+0000","last_fullsized":"2024-04-26T19:50:37.130813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997661+0000","last_change":"2024-04-26T19:50:31.113048+0000","last_active":"2024-04-26T19:50:37.997661+0000","last_peered":"2024-04-26T19:50:37.997661+0000","last_clean":"2024-04-26T19:50:37.997661+0000","last_became_active":"2024-04-26T19:50:31.112760+0000","last_became_peered":"2024-04-26T19:50:31.112760+0000","last_unstale":"2024-04-26T19:50:37.997661+0000","last_undegraded":"2024-04-26T19:50:37.997661+0000","last_fullsized":"2024-04-26T19:50:37.997661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653673+0000","last_change":"2024-04-26T19:50:31.116289+0000","last_active":"2024-04-26T19:50:38.653673+0000","last_peered":"2024-04-26T19:50:38.653673+0000","last_clean":"2024-04-26T19:50:38.653673+0000","last_became_active":"2024-04-26T19:50:31.116030+0000","last_became_peered":"2024-04-26T19:50:31.116030+0000","last_unstale":"2024-04-26T19:50:38.653673+0000","last_undegraded":"2024-04-26T19:50:38.653673+0000","last_fullsized":"2024-04-26T19:50:38.653673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130946+0000","last_change":"2024-04-26T19:50:31.106460+0000","last_active":"2024-04-26T19:50:37.130946+0000","last_peered":"2024-04-26T19:50:37.130946+0000","last_clean":"2024-04-26T19:50:37.130946+0000","last_became_active":"2024-04-26T19:50:31.106066+0000","last_became_peered":"2024-04-26T19:50:31.106066+0000","last_unstale":"2024-04-26T19:50:37.130946+0000","last_undegraded":"2024-04-26T19:50:37.130946+0000","last_fullsized":"2024-04-26T19:50:37.130946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652449+0000","last_change":"2024-04-26T19:50:31.121061+0000","last_active":"2024-04-26T19:50:38.652449+0000","last_peered":"2024-04-26T19:50:38.652449+0000","last_clean":"2024-04-26T19:50:38.652449+0000","last_became_active":"2024-04-26T19:50:31.120795+0000","last_became_peered":"2024-04-26T19:50:31.120795+0000","last_unstale":"2024-04-26T19:50:38.652449+0000","last_undegraded":"2024-04-26T19:50:38.652449+0000","last_fullsized":"2024-04-26T19:50:38.652449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130353+0000","last_change":"2024-04-26T19:50:31.108437+0000","last_active":"2024-04-26T19:50:37.130353+0000","last_peered":"2024-04-26T19:50:37.130353+0000","last_clean":"2024-04-26T19:50:37.130353+0000","last_became_active":"2024-04-26T19:50:31.108163+0000","last_became_peered":"2024-04-26T19:50:31.108163+0000","last_unstale":"2024-04-26T19:50:37.130353+0000","last_undegraded":"2024-04-26T19:50:37.130353+0000","last_fullsized":"2024-04-26T19:50:37.130353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654157+0000","last_change":"2024-04-26T19:50:31.117685+0000","last_active":"2024-04-26T19:50:38.654157+0000","last_peered":"2024-04-26T19:50:38.654157+0000","last_clean":"2024-04-26T19:50:38.654157+0000","last_became_active":"2024-04-26T19:50:31.115588+0000","last_became_peered":"2024-04-26T19:50:31.115588+0000","last_unstale":"2024-04-26T19:50:38.654157+0000","last_undegraded":"2024-04-26T19:50:38.654157+0000","last_fullsized":"2024-04-26T19:50:38.654157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:10.399495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299776+0000","last_change":"2024-04-26T19:50:31.115273+0000","last_active":"2024-04-26T19:50:38.299776+0000","last_peered":"2024-04-26T19:50:38.299776+0000","last_clean":"2024-04-26T19:50:38.299776+0000","last_became_active":"2024-04-26T19:50:31.114772+0000","last_became_peered":"2024-04-26T19:50:31.114772+0000","last_unstale":"2024-04-26T19:50:38.299776+0000","last_undegraded":"2024-04-26T19:50:38.299776+0000","last_fullsized":"2024-04-26T19:50:38.299776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653493+0000","last_change":"2024-04-26T19:50:31.110492+0000","last_active":"2024-04-26T19:50:38.653493+0000","last_peered":"2024-04-26T19:50:38.653493+0000","last_clean":"2024-04-26T19:50:38.653493+0000","last_became_active":"2024-04-26T19:50:31.110280+0000","last_became_peered":"2024-04-26T19:50:31.110280+0000","last_unstale":"2024-04-26T19:50:38.653493+0000","last_undegraded":"2024-04-26T19:50:38.653493+0000","last_fullsized":"2024-04-26T19:50:38.653493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653886+0000","last_change":"2024-04-26T19:50:31.114136+0000","last_active":"2024-04-26T19:50:38.653886+0000","last_peered":"2024-04-26T19:50:38.653886+0000","last_clean":"2024-04-26T19:50:38.653886+0000","last_became_active":"2024-04-26T19:50:31.113175+0000","last_became_peered":"2024-04-26T19:50:31.113175+0000","last_unstale":"2024-04-26T19:50:38.653886+0000","last_undegraded":"2024-04-26T19:50:38.653886+0000","last_fullsized":"2024-04-26T19:50:38.653886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:39.941264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537153+0000","last_change":"2024-04-26T19:50:31.108339+0000","last_active":"2024-04-26T19:50:38.537153+0000","last_peered":"2024-04-26T19:50:38.537153+0000","last_clean":"2024-04-26T19:50:38.537153+0000","last_became_active":"2024-04-26T19:50:31.108092+0000","last_became_peered":"2024-04-26T19:50:31.108092+0000","last_unstale":"2024-04-26T19:50:38.537153+0000","last_undegraded":"2024-04-26T19:50:38.537153+0000","last_fullsized":"2024-04-26T19:50:38.537153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:14.980411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653311+0000","last_change":"2024-04-26T19:50:31.116207+0000","last_active":"2024-04-26T19:50:38.653311+0000","last_peered":"2024-04-26T19:50:38.653311+0000","last_clean":"2024-04-26T19:50:38.653311+0000","last_became_active":"2024-04-26T19:50:31.115498+0000","last_became_peered":"2024-04-26T19:50:31.115498+0000","last_unstale":"2024-04-26T19:50:38.653311+0000","last_undegraded":"2024-04-26T19:50:38.653311+0000","last_fullsized":"2024-04-26T19:50:38.653311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:42.558104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652346+0000","last_change":"2024-04-26T19:50:31.117352+0000","last_active":"2024-04-26T19:50:38.652346+0000","last_peered":"2024-04-26T19:50:38.652346+0000","last_clean":"2024-04-26T19:50:38.652346+0000","last_became_active":"2024-04-26T19:50:31.117098+0000","last_became_peered":"2024-04-26T19:50:31.117098+0000","last_unstale":"2024-04-26T19:50:38.652346+0000","last_undegraded":"2024-04-26T19:50:38.652346+0000","last_fullsized":"2024-04-26T19:50:38.652346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997376+0000","last_change":"2024-04-26T19:50:31.113884+0000","last_active":"2024-04-26T19:50:37.997376+0000","last_peered":"2024-04-26T19:50:37.997376+0000","last_clean":"2024-04-26T19:50:37.997376+0000","last_became_active":"2024-04-26T19:50:31.113607+0000","last_became_peered":"2024-04-26T19:50:31.113607+0000","last_unstale":"2024-04-26T19:50:37.997376+0000","last_undegraded":"2024-04-26T19:50:37.997376+0000","last_fullsized":"2024-04-26T19:50:37.997376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299710+0000","last_change":"2024-04-26T19:50:31.110086+0000","last_active":"2024-04-26T19:50:38.299710+0000","last_peered":"2024-04-26T19:50:38.299710+0000","last_clean":"2024-04-26T19:50:38.299710+0000","last_became_active":"2024-04-26T19:50:31.109800+0000","last_became_peered":"2024-04-26T19:50:31.109800+0000","last_unstale":"2024-04-26T19:50:38.299710+0000","last_undegraded":"2024-04-26T19:50:38.299710+0000","last_fullsized":"2024-04-26T19:50:38.299710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:45.295846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130514+0000","last_change":"2024-04-26T19:50:31.109078+0000","last_active":"2024-04-26T19:50:37.130514+0000","last_peered":"2024-04-26T19:50:37.130514+0000","last_clean":"2024-04-26T19:50:37.130514+0000","last_became_active":"2024-04-26T19:50:31.107801+0000","last_became_peered":"2024-04-26T19:50:31.107801+0000","last_unstale":"2024-04-26T19:50:37.130514+0000","last_undegraded":"2024-04-26T19:50:37.130514+0000","last_fullsized":"2024-04-26T19:50:37.130514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130138+0000","last_change":"2024-04-26T19:50:31.103841+0000","last_active":"2024-04-26T19:50:37.130138+0000","last_peered":"2024-04-26T19:50:37.130138+0000","last_clean":"2024-04-26T19:50:37.130138+0000","last_became_active":"2024-04-26T19:50:31.103366+0000","last_became_peered":"2024-04-26T19:50:31.103366+0000","last_unstale":"2024-04-26T19:50:37.130138+0000","last_undegraded":"2024-04-26T19:50:37.130138+0000","last_fullsized":"2024-04-26T19:50:37.130138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:22.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653217+0000","last_change":"2024-04-26T19:50:31.113740+0000","last_active":"2024-04-26T19:50:38.653217+0000","last_peered":"2024-04-26T19:50:38.653217+0000","last_clean":"2024-04-26T19:50:38.653217+0000","last_became_active":"2024-04-26T19:50:31.113431+0000","last_became_peered":"2024-04-26T19:50:31.113431+0000","last_unstale":"2024-04-26T19:50:38.653217+0000","last_undegraded":"2024-04-26T19:50:38.653217+0000","last_fullsized":"2024-04-26T19:50:38.653217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653594+0000","last_change":"2024-04-26T19:50:31.113198+0000","last_active":"2024-04-26T19:50:38.653594+0000","last_peered":"2024-04-26T19:50:38.653594+0000","last_clean":"2024-04-26T19:50:38.653594+0000","last_became_active":"2024-04-26T19:50:31.112582+0000","last_became_peered":"2024-04-26T19:50:31.112582+0000","last_unstale":"2024-04-26T19:50:38.653594+0000","last_undegraded":"2024-04-26T19:50:38.653594+0000","last_fullsized":"2024-04-26T19:50:38.653594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653995+0000","last_change":"2024-04-26T19:50:31.120475+0000","last_active":"2024-04-26T19:50:38.653995+0000","last_peered":"2024-04-26T19:50:38.653995+0000","last_clean":"2024-04-26T19:50:38.653995+0000","last_became_active":"2024-04-26T19:50:31.118576+0000","last_became_peered":"2024-04-26T19:50:31.118576+0000","last_unstale":"2024-04-26T19:50:38.653995+0000","last_undegraded":"2024-04-26T19:50:38.653995+0000","last_fullsized":"2024-04-26T19:50:38.653995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.291653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130241+0000","last_change":"2024-04-26T19:50:31.111907+0000","last_active":"2024-04-26T19:50:37.130241+0000","last_peered":"2024-04-26T19:50:37.130241+0000","last_clean":"2024-04-26T19:50:37.130241+0000","last_became_active":"2024-04-26T19:50:31.111623+0000","last_became_peered":"2024-04-26T19:50:31.111623+0000","last_unstale":"2024-04-26T19:50:37.130241+0000","last_undegraded":"2024-04-26T19:50:37.130241+0000","last_fullsized":"2024-04-26T19:50:37.130241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652974+0000","last_change":"2024-04-26T19:50:31.112347+0000","last_active":"2024-04-26T19:50:38.652974+0000","last_peered":"2024-04-26T19:50:38.652974+0000","last_clean":"2024-04-26T19:50:38.652974+0000","last_became_active":"2024-04-26T19:50:31.112074+0000","last_became_peered":"2024-04-26T19:50:31.112074+0000","last_unstale":"2024-04-26T19:50:38.652974+0000","last_undegraded":"2024-04-26T19:50:38.652974+0000","last_fullsized":"2024-04-26T19:50:38.652974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654050+0000","last_change":"2024-04-26T19:50:31.115255+0000","last_active":"2024-04-26T19:50:38.654050+0000","last_peered":"2024-04-26T19:50:38.654050+0000","last_clean":"2024-04-26T19:50:38.654050+0000","last_became_active":"2024-04-26T19:50:31.114991+0000","last_became_peered":"2024-04-26T19:50:31.114991+0000","last_unstale":"2024-04-26T19:50:38.654050+0000","last_undegraded":"2024-04-26T19:50:38.654050+0000","last_fullsized":"2024-04-26T19:50:38.654050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:39.941264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653558+0000","last_change":"2024-04-26T19:50:31.111763+0000","last_active":"2024-04-26T19:50:38.653558+0000","last_peered":"2024-04-26T19:50:38.653558+0000","last_clean":"2024-04-26T19:50:38.653558+0000","last_became_active":"2024-04-26T19:50:31.111487+0000","last_became_peered":"2024-04-26T19:50:31.111487+0000","last_unstale":"2024-04-26T19:50:38.653558+0000","last_undegraded":"2024-04-26T19:50:38.653558+0000","last_fullsized":"2024-04-26T19:50:38.653558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:13.577254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652454+0000","last_change":"2024-04-26T19:50:31.120637+0000","last_active":"2024-04-26T19:50:38.652454+0000","last_peered":"2024-04-26T19:50:38.652454+0000","last_clean":"2024-04-26T19:50:38.652454+0000","last_became_active":"2024-04-26T19:50:31.120388+0000","last_became_peered":"2024-04-26T19:50:31.120388+0000","last_unstale":"2024-04-26T19:50:38.652454+0000","last_undegraded":"2024-04-26T19:50:38.652454+0000","last_fullsized":"2024-04-26T19:50:38.652454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299955+0000","last_change":"2024-04-26T19:50:31.104484+0000","last_active":"2024-04-26T19:50:38.299955+0000","last_peered":"2024-04-26T19:50:38.299955+0000","last_clean":"2024-04-26T19:50:38.299955+0000","last_became_active":"2024-04-26T19:50:31.104172+0000","last_became_peered":"2024-04-26T19:50:31.104172+0000","last_unstale":"2024-04-26T19:50:38.299955+0000","last_undegraded":"2024-04-26T19:50:38.299955+0000","last_fullsized":"2024-04-26T19:50:38.299955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997884+0000","last_change":"2024-04-26T19:50:31.110923+0000","last_active":"2024-04-26T19:50:37.997884+0000","last_peered":"2024-04-26T19:50:37.997884+0000","last_clean":"2024-04-26T19:50:37.997884+0000","last_became_active":"2024-04-26T19:50:31.110601+0000","last_became_peered":"2024-04-26T19:50:31.110601+0000","last_unstale":"2024-04-26T19:50:37.997884+0000","last_undegraded":"2024-04-26T19:50:37.997884+0000","last_fullsized":"2024-04-26T19:50:37.997884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652978+0000","last_change":"2024-04-26T19:50:31.112465+0000","last_active":"2024-04-26T19:50:38.652978+0000","last_peered":"2024-04-26T19:50:38.652978+0000","last_clean":"2024-04-26T19:50:38.652978+0000","last_became_active":"2024-04-26T19:50:31.112186+0000","last_became_peered":"2024-04-26T19:50:31.112186+0000","last_unstale":"2024-04-26T19:50:38.652978+0000","last_undegraded":"2024-04-26T19:50:38.652978+0000","last_fullsized":"2024-04-26T19:50:38.652978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:10.399495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:50:58.512 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:50:58.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.643+0000 7fe4a3025640 1 Processor -- start 2024-04-26T19:50:58.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.643+0000 7fe4a3025640 1 -- start start 2024-04-26T19:50:58.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.643+0000 7fe4a3025640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe49c160770 0x7fe49c1637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.643+0000 7fe4a3025640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49c144ca0 con 0x7fe49c0b4220 2024-04-26T19:50:58.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.643+0000 7fe4a3025640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe49c163e00 con 0x7fe49c160390 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.643+0000 7fe4a3025640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe49c144f30 con 0x7fe49c160770 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49bfff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe49c0b4220 0x7fe49c1630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53090/0 (socket says 172.21.15.79:53090) 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49bfff640 1 -- 172.21.15.79:0/3406653118 learned_addr learned my addr 172.21.15.79:0/3406653118 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe4a15c2640 1 --2- 172.21.15.79:0/3406653118 >> v2:172.21.15.83:3300/0 conn(0x7fe49c160770 0x7fe49c1637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2929592438 0 0) 0x7fe49c144ca0 con 0x7fe49c0b4220 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe484003520 con 0x7fe49c0b4220 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 773103695 0 0) 0x7fe484003520 con 0x7fe49c0b4220 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 >> v2:172.21.15.83:3300/0 conn(0x7fe49c160770 msgr2=0x7fe49c1637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 --2- 172.21.15.79:0/3406653118 >> v2:172.21.15.83:3300/0 conn(0x7fe49c160770 0x7fe49c1637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 >> v1:172.21.15.83:6789/0 conn(0x7fe49c160390 legacy=0x7fe49c0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c145210 con 0x7fe49c0b4220 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fe490001670 con 0x7fe49c0b4220 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 >> v1:172.21.15.79:6789/0 conn(0x7fe49c0b4220 legacy=0x7fe49c1630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4840039b0 0x7fe484003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe484004800 0x7fe484004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe48400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c144f30 con 0x7fe4840051c0 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe48400c160 con 0x7fe484004800 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe48400c360 con 0x7fe4840039b0 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe4a15c2640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe48400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe49bfff640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe484004800 0x7fe484004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe4a0dc1640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4840039b0 0x7fe484003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe4a15c2640 1 -- 172.21.15.79:0/3406653118 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4840039b0 msgr2=0x7fe484003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe4a15c2640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4840039b0 0x7fe484003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe4a15c2640 1 -- 172.21.15.79:0/3406653118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe484004800 msgr2=0x7fe484004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe4a15c2640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe484004800 0x7fe484004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.644+0000 7fe4a15c2640 1 -- 172.21.15.79:0/3406653118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe484003520 con 0x7fe4840051c0 2024-04-26T19:50:58.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.645+0000 7fe4a15c2640 1 -- 172.21.15.79:0/3406653118 reap_dead start 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.645+0000 7fe4a15c2640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe48400b9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe494009fc0 tx=0x7fe484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3955109d1d1b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.645+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494035020 con 0x7fe4840051c0 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.645+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe494003b90 con 0x7fe4840051c0 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.645+0000 7fe49b7fe640 1 -- 172.21.15.79:0/3406653118 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494009c10 con 0x7fe4840051c0 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.646+0000 7fe4a3025640 1 -- 172.21.15.79:0/3406653118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 msgr2=0x7fe48400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.646+0000 7fe4a3025640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe48400b9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe494009fc0 tx=0x7fe484005830 comp rx=0 tx=0).stop 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.646+0000 7fe4a3025640 1 -- 172.21.15.79:0/3406653118 shutdown_connections 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.646+0000 7fe4a3025640 1 --2- 172.21.15.79:0/3406653118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe48400b9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.646+0000 7fe4a3025640 1 -- 172.21.15.79:0/3406653118 >> 172.21.15.79:0/3406653118 conn(0x7fe49c0b2720 msgr2=0x7fe49c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.646+0000 7fe4a3025640 1 -- 172.21.15.79:0/3406653118 shutdown_connections 2024-04-26T19:50:58.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.646+0000 7fe4a3025640 1 -- 172.21.15.79:0/3406653118 wait complete. 2024-04-26T19:50:58.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.647+0000 7fe4a3025640 1 Processor -- start 2024-04-26T19:50:58.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.647+0000 7fe4a3025640 1 -- start start 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a3025640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe49c0d2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a3025640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe49c161600 0x7fe49c0af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a3025640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe49c161a60 0x7fe49c0af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a3025640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c145b90 con 0x7fe4840051c0 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a3025640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c145a10 con 0x7fe49c161a60 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a3025640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe49c0b3b10 con 0x7fe49c161600 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a0dc1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe49c0d2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a0dc1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe49c0d2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46246/0 (socket says 172.21.15.79:46246) 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a0dc1640 1 -- 172.21.15.79:0/3792113997 learned_addr learned my addr 172.21.15.79:0/3792113997 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe49bfff640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe49c161600 0x7fe49c0af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a15c2640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe49c161a60 0x7fe49c0af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a0dc1640 1 -- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe49c161600 msgr2=0x7fe49c0af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a0dc1640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe49c161600 0x7fe49c0af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a0dc1640 1 -- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe49c161a60 msgr2=0x7fe49c0af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a0dc1640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe49c161a60 0x7fe49c0af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a0dc1640 1 -- 172.21.15.79:0/3792113997 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c162580 con 0x7fe4840051c0 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe49bfff640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe49c161600 0x7fe49c0af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a15c2640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe49c161a60 0x7fe49c0af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4a0dc1640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe49c0d2060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00f650 tx=0x7fe48c00fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.648+0000 7fe4997fa640 1 -- 172.21.15.79:0/3792113997 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0050d0 con 0x7fe4840051c0 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.649+0000 7fe4997fa640 1 -- 172.21.15.79:0/3792113997 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48c018070 con 0x7fe4840051c0 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.649+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c161fc0 con 0x7fe4840051c0 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.649+0000 7fe4997fa640 1 -- 172.21.15.79:0/3792113997 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c005490 con 0x7fe4840051c0 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.649+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c15ab70 con 0x7fe4840051c0 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.650+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c0506c0 con 0x7fe4840051c0 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.650+0000 7fe4997fa640 1 -- 172.21.15.79:0/3792113997 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe48c00d6d0 con 0x7fe4840051c0 2024-04-26T19:50:58.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.650+0000 7fe4997fa640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe48806c8f0 0x7fe48806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.651+0000 7fe4997fa640 1 -- 172.21.15.79:0/3792113997 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe48c08bbe0 con 0x7fe4840051c0 2024-04-26T19:50:58.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.651+0000 7fe49bfff640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe48806c8f0 0x7fe48806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.651+0000 7fe49bfff640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe48806c8f0 0x7fe48806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe49c1595c0 tx=0x7fe490009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:58.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.655+0000 7fe4997fa640 1 -- 172.21.15.79:0/3792113997 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe48c059cd0 con 0x7fe4840051c0 2024-04-26T19:50:58.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.829+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe49c0b2500 con 0x7fe48806c8f0 2024-04-26T19:50:58.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.836+0000 7fe4997fa640 1 -- 172.21.15.79:0/3792113997 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332533 (secure 0 0 0) 0x7fe49c0b2500 con 0x7fe48806c8f0 2024-04-26T19:50:58.840 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:58.842 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe48806c8f0 msgr2=0x7fe48806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe48806c8f0 0x7fe48806edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe49c1595c0 tx=0x7fe490009040 comp rx=0 tx=0).stop 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 msgr2=0x7fe49c0d2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe49c0d2060 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00f650 tx=0x7fe48c00fb10 comp rx=0 tx=0).stop 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 shutdown_connections 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe48806c8f0 0x7fe48806edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe49c161a60 0x7fe49c0af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe49c161600 0x7fe49c0af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 --2- 172.21.15.79:0/3792113997 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4840051c0 0x7fe49c0d2060 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 >> 172.21.15.79:0/3792113997 conn(0x7fe49c0b2720 msgr2=0x7fe49c06e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 shutdown_connections 2024-04-26T19:50:58.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.840+0000 7fe4a3025640 1 -- 172.21.15.79:0/3792113997 wait complete. 2024-04-26T19:50:58.852 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-26T19:50:58.258498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219460,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755260,"statfs":{"total":773094113280,"available":772869386240,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003297"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300284+0000","last_change":"2024-04-26T19:50:30.105908+0000","last_active":"2024-04-26T19:50:38.300284+0000","last_peered":"2024-04-26T19:50:38.300284+0000","last_clean":"2024-04-26T19:50:38.300284+0000","last_became_active":"2024-04-26T19:50:30.105650+0000","last_became_peered":"2024-04-26T19:50:30.105650+0000","last_unstale":"2024-04-26T19:50:38.300284+0000","last_undegraded":"2024-04-26T19:50:38.300284+0000","last_fullsized":"2024-04-26T19:50:38.300284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:33:25.896804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131172+0000","last_change":"2024-04-26T19:50:31.110936+0000","last_active":"2024-04-26T19:50:37.131172+0000","last_peered":"2024-04-26T19:50:37.131172+0000","last_clean":"2024-04-26T19:50:37.131172+0000","last_became_active":"2024-04-26T19:50:31.110602+0000","last_became_peered":"2024-04-26T19:50:31.110602+0000","last_unstale":"2024-04-26T19:50:37.131172+0000","last_undegraded":"2024-04-26T19:50:37.131172+0000","last_fullsized":"2024-04-26T19:50:37.131172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130254+0000","last_change":"2024-04-26T19:50:35.626950+0000","last_active":"2024-04-26T19:50:37.130254+0000","last_peered":"2024-04-26T19:50:37.130254+0000","last_clean":"2024-04-26T19:50:37.130254+0000","last_became_active":"2024-04-26T19:50:35.626309+0000","last_became_peered":"2024-04-26T19:50:35.626309+0000","last_unstale":"2024-04-26T19:50:37.130254+0000","last_undegraded":"2024-04-26T19:50:37.130254+0000","last_fullsized":"2024-04-26T19:50:37.130254+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:30:42.055139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131089+0000","last_change":"2024-04-26T19:50:30.112742+0000","last_active":"2024-04-26T19:50:37.131089+0000","last_peered":"2024-04-26T19:50:37.131089+0000","last_clean":"2024-04-26T19:50:37.131089+0000","last_became_active":"2024-04-26T19:50:30.112460+0000","last_became_peered":"2024-04-26T19:50:30.112460+0000","last_unstale":"2024-04-26T19:50:37.131089+0000","last_undegraded":"2024-04-26T19:50:37.131089+0000","last_fullsized":"2024-04-26T19:50:37.131089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:21.153340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300125+0000","last_change":"2024-04-26T19:50:31.114135+0000","last_active":"2024-04-26T19:50:38.300125+0000","last_peered":"2024-04-26T19:50:38.300125+0000","last_clean":"2024-04-26T19:50:38.300125+0000","last_became_active":"2024-04-26T19:50:31.113834+0000","last_became_peered":"2024-04-26T19:50:31.113834+0000","last_unstale":"2024-04-26T19:50:38.300125+0000","last_undegraded":"2024-04-26T19:50:38.300125+0000","last_fullsized":"2024-04-26T19:50:38.300125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129662+0000","last_change":"2024-04-26T19:50:35.141805+0000","last_active":"2024-04-26T19:50:37.129662+0000","last_peered":"2024-04-26T19:50:37.129662+0000","last_clean":"2024-04-26T19:50:37.129662+0000","last_became_active":"2024-04-26T19:50:35.141362+0000","last_became_peered":"2024-04-26T19:50:35.141362+0000","last_unstale":"2024-04-26T19:50:37.129662+0000","last_undegraded":"2024-04-26T19:50:37.129662+0000","last_fullsized":"2024-04-26T19:50:37.129662+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:10.325286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653474+0000","last_change":"2024-04-26T19:50:30.107616+0000","last_active":"2024-04-26T19:50:38.653474+0000","last_peered":"2024-04-26T19:50:38.653474+0000","last_clean":"2024-04-26T19:50:38.653474+0000","last_became_active":"2024-04-26T19:50:30.107304+0000","last_became_peered":"2024-04-26T19:50:30.107304+0000","last_unstale":"2024-04-26T19:50:38.653474+0000","last_undegraded":"2024-04-26T19:50:38.653474+0000","last_fullsized":"2024-04-26T19:50:38.653474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:12.144175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652574+0000","last_change":"2024-04-26T19:50:31.114725+0000","last_active":"2024-04-26T19:50:38.652574+0000","last_peered":"2024-04-26T19:50:38.652574+0000","last_clean":"2024-04-26T19:50:38.652574+0000","last_became_active":"2024-04-26T19:50:31.114444+0000","last_became_peered":"2024-04-26T19:50:31.114444+0000","last_unstale":"2024-04-26T19:50:38.652574+0000","last_undegraded":"2024-04-26T19:50:38.652574+0000","last_fullsized":"2024-04-26T19:50:38.652574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129888+0000","last_change":"2024-04-26T19:50:35.145133+0000","last_active":"2024-04-26T19:50:37.129888+0000","last_peered":"2024-04-26T19:50:37.129888+0000","last_clean":"2024-04-26T19:50:37.129888+0000","last_became_active":"2024-04-26T19:50:35.144310+0000","last_became_peered":"2024-04-26T19:50:35.144310+0000","last_unstale":"2024-04-26T19:50:37.129888+0000","last_undegraded":"2024-04-26T19:50:37.129888+0000","last_fullsized":"2024-04-26T19:50:37.129888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:35.220332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652453+0000","last_change":"2024-04-26T19:50:30.115029+0000","last_active":"2024-04-26T19:50:38.652453+0000","last_peered":"2024-04-26T19:50:38.652453+0000","last_clean":"2024-04-26T19:50:38.652453+0000","last_became_active":"2024-04-26T19:50:30.114719+0000","last_became_peered":"2024-04-26T19:50:30.114719+0000","last_unstale":"2024-04-26T19:50:38.652453+0000","last_undegraded":"2024-04-26T19:50:38.652453+0000","last_fullsized":"2024-04-26T19:50:38.652453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:16:28.557299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654620+0000","last_change":"2024-04-26T19:50:31.114802+0000","last_active":"2024-04-26T19:50:38.654620+0000","last_peered":"2024-04-26T19:50:38.654620+0000","last_clean":"2024-04-26T19:50:38.654620+0000","last_became_active":"2024-04-26T19:50:31.114489+0000","last_became_peered":"2024-04-26T19:50:31.114489+0000","last_unstale":"2024-04-26T19:50:38.654620+0000","last_undegraded":"2024-04-26T19:50:38.654620+0000","last_fullsized":"2024-04-26T19:50:38.654620+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129602+0000","last_change":"2024-04-26T19:50:35.147238+0000","last_active":"2024-04-26T19:50:37.129602+0000","last_peered":"2024-04-26T19:50:37.129602+0000","last_clean":"2024-04-26T19:50:37.129602+0000","last_became_active":"2024-04-26T19:50:35.146793+0000","last_became_peered":"2024-04-26T19:50:35.146793+0000","last_unstale":"2024-04-26T19:50:37.129602+0000","last_undegraded":"2024-04-26T19:50:37.129602+0000","last_fullsized":"2024-04-26T19:50:37.129602+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:14.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654487+0000","last_change":"2024-04-26T19:50:31.104842+0000","last_active":"2024-04-26T19:50:38.654487+0000","last_peered":"2024-04-26T19:50:38.654487+0000","last_clean":"2024-04-26T19:50:38.654487+0000","last_became_active":"2024-04-26T19:50:31.104587+0000","last_became_peered":"2024-04-26T19:50:31.104587+0000","last_unstale":"2024-04-26T19:50:38.654487+0000","last_undegraded":"2024-04-26T19:50:38.654487+0000","last_fullsized":"2024-04-26T19:50:38.654487+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653208+0000","last_change":"2024-04-26T19:50:30.112050+0000","last_active":"2024-04-26T19:50:38.653208+0000","last_peered":"2024-04-26T19:50:38.653208+0000","last_clean":"2024-04-26T19:50:38.653208+0000","last_became_active":"2024-04-26T19:50:30.111754+0000","last_became_peered":"2024-04-26T19:50:30.111754+0000","last_unstale":"2024-04-26T19:50:38.653208+0000","last_undegraded":"2024-04-26T19:50:38.653208+0000","last_fullsized":"2024-04-26T19:50:38.653208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130351+0000","last_change":"2024-04-26T19:50:35.145983+0000","last_active":"2024-04-26T19:50:37.130351+0000","last_peered":"2024-04-26T19:50:37.130351+0000","last_clean":"2024-04-26T19:50:37.130351+0000","last_became_active":"2024-04-26T19:50:35.144815+0000","last_became_peered":"2024-04-26T19:50:35.144815+0000","last_unstale":"2024-04-26T19:50:37.130351+0000","last_undegraded":"2024-04-26T19:50:37.130351+0000","last_fullsized":"2024-04-26T19:50:37.130351+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130515+0000","last_change":"2024-04-26T19:50:31.109267+0000","last_active":"2024-04-26T19:50:37.130515+0000","last_peered":"2024-04-26T19:50:37.130515+0000","last_clean":"2024-04-26T19:50:37.130515+0000","last_became_active":"2024-04-26T19:50:31.108771+0000","last_became_peered":"2024-04-26T19:50:31.108771+0000","last_unstale":"2024-04-26T19:50:37.130515+0000","last_undegraded":"2024-04-26T19:50:37.130515+0000","last_fullsized":"2024-04-26T19:50:37.130515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997666+0000","last_change":"2024-04-26T19:50:30.110310+0000","last_active":"2024-04-26T19:50:37.997666+0000","last_peered":"2024-04-26T19:50:37.997666+0000","last_clean":"2024-04-26T19:50:37.997666+0000","last_became_active":"2024-04-26T19:50:30.109819+0000","last_became_peered":"2024-04-26T19:50:30.109819+0000","last_unstale":"2024-04-26T19:50:37.997666+0000","last_undegraded":"2024-04-26T19:50:37.997666+0000","last_fullsized":"2024-04-26T19:50:37.997666+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:41.838039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130058+0000","last_change":"2024-04-26T19:50:35.143564+0000","last_active":"2024-04-26T19:50:37.130058+0000","last_peered":"2024-04-26T19:50:37.130058+0000","last_clean":"2024-04-26T19:50:37.130058+0000","last_became_active":"2024-04-26T19:50:35.143119+0000","last_became_peered":"2024-04-26T19:50:35.143119+0000","last_unstale":"2024-04-26T19:50:37.130058+0000","last_undegraded":"2024-04-26T19:50:37.130058+0000","last_fullsized":"2024-04-26T19:50:37.130058+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:11.052818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130395+0000","last_change":"2024-04-26T19:50:31.112367+0000","last_active":"2024-04-26T19:50:37.130395+0000","last_peered":"2024-04-26T19:50:37.130395+0000","last_clean":"2024-04-26T19:50:37.130395+0000","last_became_active":"2024-04-26T19:50:31.112070+0000","last_became_peered":"2024-04-26T19:50:31.112070+0000","last_unstale":"2024-04-26T19:50:37.130395+0000","last_undegraded":"2024-04-26T19:50:37.130395+0000","last_fullsized":"2024-04-26T19:50:37.130395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997401+0000","last_change":"2024-04-26T19:50:30.105988+0000","last_active":"2024-04-26T19:50:37.997401+0000","last_peered":"2024-04-26T19:50:37.997401+0000","last_clean":"2024-04-26T19:50:37.997401+0000","last_became_active":"2024-04-26T19:50:30.103503+0000","last_became_peered":"2024-04-26T19:50:30.103503+0000","last_unstale":"2024-04-26T19:50:37.997401+0000","last_undegraded":"2024-04-26T19:50:37.997401+0000","last_fullsized":"2024-04-26T19:50:37.997401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997963+0000","last_change":"2024-04-26T19:50:35.626255+0000","last_active":"2024-04-26T19:50:37.997963+0000","last_peered":"2024-04-26T19:50:37.997963+0000","last_clean":"2024-04-26T19:50:37.997963+0000","last_became_active":"2024-04-26T19:50:35.625845+0000","last_became_peered":"2024-04-26T19:50:35.625845+0000","last_unstale":"2024-04-26T19:50:37.997963+0000","last_undegraded":"2024-04-26T19:50:37.997963+0000","last_fullsized":"2024-04-26T19:50:37.997963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:02.874670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129835+0000","last_change":"2024-04-26T19:50:31.106338+0000","last_active":"2024-04-26T19:50:37.129835+0000","last_peered":"2024-04-26T19:50:37.129835+0000","last_clean":"2024-04-26T19:50:37.129835+0000","last_became_active":"2024-04-26T19:50:31.106097+0000","last_became_peered":"2024-04-26T19:50:31.106097+0000","last_unstale":"2024-04-26T19:50:37.129835+0000","last_undegraded":"2024-04-26T19:50:37.129835+0000","last_fullsized":"2024-04-26T19:50:37.129835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:13.140788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654064+0000","last_change":"2024-04-26T19:50:30.114014+0000","last_active":"2024-04-26T19:50:38.654064+0000","last_peered":"2024-04-26T19:50:38.654064+0000","last_clean":"2024-04-26T19:50:38.654064+0000","last_became_active":"2024-04-26T19:50:30.113720+0000","last_became_peered":"2024-04-26T19:50:30.113720+0000","last_unstale":"2024-04-26T19:50:38.654064+0000","last_undegraded":"2024-04-26T19:50:38.654064+0000","last_fullsized":"2024-04-26T19:50:38.654064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130140+0000","last_change":"2024-04-26T19:50:35.144698+0000","last_active":"2024-04-26T19:50:37.130140+0000","last_peered":"2024-04-26T19:50:37.130140+0000","last_clean":"2024-04-26T19:50:37.130140+0000","last_became_active":"2024-04-26T19:50:35.144169+0000","last_became_peered":"2024-04-26T19:50:35.144169+0000","last_unstale":"2024-04-26T19:50:37.130140+0000","last_undegraded":"2024-04-26T19:50:37.130140+0000","last_fullsized":"2024-04-26T19:50:37.130140+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300094+0000","last_change":"2024-04-26T19:50:31.105626+0000","last_active":"2024-04-26T19:50:38.300094+0000","last_peered":"2024-04-26T19:50:38.300094+0000","last_clean":"2024-04-26T19:50:38.300094+0000","last_became_active":"2024-04-26T19:50:31.105369+0000","last_became_peered":"2024-04-26T19:50:31.105369+0000","last_unstale":"2024-04-26T19:50:38.300094+0000","last_undegraded":"2024-04-26T19:50:38.300094+0000","last_fullsized":"2024-04-26T19:50:38.300094+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654017+0000","last_change":"2024-04-26T19:50:30.104092+0000","last_active":"2024-04-26T19:50:38.654017+0000","last_peered":"2024-04-26T19:50:38.654017+0000","last_clean":"2024-04-26T19:50:38.654017+0000","last_became_active":"2024-04-26T19:50:30.103848+0000","last_became_peered":"2024-04-26T19:50:30.103848+0000","last_unstale":"2024-04-26T19:50:38.654017+0000","last_undegraded":"2024-04-26T19:50:38.654017+0000","last_fullsized":"2024-04-26T19:50:38.654017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:21.027009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129689+0000","last_change":"2024-04-26T19:50:35.624499+0000","last_active":"2024-04-26T19:50:37.129689+0000","last_peered":"2024-04-26T19:50:37.129689+0000","last_clean":"2024-04-26T19:50:37.129689+0000","last_became_active":"2024-04-26T19:50:35.624026+0000","last_became_peered":"2024-04-26T19:50:35.624026+0000","last_unstale":"2024-04-26T19:50:37.129689+0000","last_undegraded":"2024-04-26T19:50:37.129689+0000","last_fullsized":"2024-04-26T19:50:37.129689+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:46.591736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131049+0000","last_change":"2024-04-26T19:50:31.116347+0000","last_active":"2024-04-26T19:50:37.131049+0000","last_peered":"2024-04-26T19:50:37.131049+0000","last_clean":"2024-04-26T19:50:37.131049+0000","last_became_active":"2024-04-26T19:50:31.116037+0000","last_became_peered":"2024-04-26T19:50:31.116037+0000","last_unstale":"2024-04-26T19:50:37.131049+0000","last_undegraded":"2024-04-26T19:50:37.131049+0000","last_fullsized":"2024-04-26T19:50:37.131049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997937+0000","last_change":"2024-04-26T19:50:30.105663+0000","last_active":"2024-04-26T19:50:37.997937+0000","last_peered":"2024-04-26T19:50:37.997937+0000","last_clean":"2024-04-26T19:50:37.997937+0000","last_became_active":"2024-04-26T19:50:30.105387+0000","last_became_peered":"2024-04-26T19:50:30.105387+0000","last_unstale":"2024-04-26T19:50:37.997937+0000","last_undegraded":"2024-04-26T19:50:37.997937+0000","last_fullsized":"2024-04-26T19:50:37.997937+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130198+0000","last_change":"2024-04-26T19:50:35.625444+0000","last_active":"2024-04-26T19:50:37.130198+0000","last_peered":"2024-04-26T19:50:37.130198+0000","last_clean":"2024-04-26T19:50:37.130198+0000","last_became_active":"2024-04-26T19:50:35.625054+0000","last_became_peered":"2024-04-26T19:50:35.625054+0000","last_unstale":"2024-04-26T19:50:37.130198+0000","last_undegraded":"2024-04-26T19:50:37.130198+0000","last_fullsized":"2024-04-26T19:50:37.130198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:21:07.422780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997757+0000","last_change":"2024-04-26T19:50:31.109145+0000","last_active":"2024-04-26T19:50:37.997757+0000","last_peered":"2024-04-26T19:50:37.997757+0000","last_clean":"2024-04-26T19:50:37.997757+0000","last_became_active":"2024-04-26T19:50:31.108812+0000","last_became_peered":"2024-04-26T19:50:31.108812+0000","last_unstale":"2024-04-26T19:50:37.997757+0000","last_undegraded":"2024-04-26T19:50:37.997757+0000","last_fullsized":"2024-04-26T19:50:37.997757+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653987+0000","last_change":"2024-04-26T19:50:30.112025+0000","last_active":"2024-04-26T19:50:38.653987+0000","last_peered":"2024-04-26T19:50:38.653987+0000","last_clean":"2024-04-26T19:50:38.653987+0000","last_became_active":"2024-04-26T19:50:30.111777+0000","last_became_peered":"2024-04-26T19:50:30.111777+0000","last_unstale":"2024-04-26T19:50:38.653987+0000","last_undegraded":"2024-04-26T19:50:38.653987+0000","last_fullsized":"2024-04-26T19:50:38.653987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:41.838039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299918+0000","last_change":"2024-04-26T19:50:35.623970+0000","last_active":"2024-04-26T19:50:38.299918+0000","last_peered":"2024-04-26T19:50:38.299918+0000","last_clean":"2024-04-26T19:50:38.299918+0000","last_became_active":"2024-04-26T19:50:35.623472+0000","last_became_peered":"2024-04-26T19:50:35.623472+0000","last_unstale":"2024-04-26T19:50:38.299918+0000","last_undegraded":"2024-04-26T19:50:38.299918+0000","last_fullsized":"2024-04-26T19:50:38.299918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654678+0000","last_change":"2024-04-26T19:50:31.120338+0000","last_active":"2024-04-26T19:50:38.654678+0000","last_peered":"2024-04-26T19:50:38.654678+0000","last_clean":"2024-04-26T19:50:38.654678+0000","last_became_active":"2024-04-26T19:50:31.120074+0000","last_became_peered":"2024-04-26T19:50:31.120074+0000","last_unstale":"2024-04-26T19:50:38.654678+0000","last_undegraded":"2024-04-26T19:50:38.654678+0000","last_fullsized":"2024-04-26T19:50:38.654678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997828+0000","last_change":"2024-04-26T19:50:30.106832+0000","last_active":"2024-04-26T19:50:37.997828+0000","last_peered":"2024-04-26T19:50:37.997828+0000","last_clean":"2024-04-26T19:50:37.997828+0000","last_became_active":"2024-04-26T19:50:30.106553+0000","last_became_peered":"2024-04-26T19:50:30.106553+0000","last_unstale":"2024-04-26T19:50:37.997828+0000","last_undegraded":"2024-04-26T19:50:37.997828+0000","last_fullsized":"2024-04-26T19:50:37.997828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997421+0000","last_change":"2024-04-26T19:50:35.149679+0000","last_active":"2024-04-26T19:50:37.997421+0000","last_peered":"2024-04-26T19:50:37.997421+0000","last_clean":"2024-04-26T19:50:37.997421+0000","last_became_active":"2024-04-26T19:50:35.149304+0000","last_became_peered":"2024-04-26T19:50:35.149304+0000","last_unstale":"2024-04-26T19:50:37.997421+0000","last_undegraded":"2024-04-26T19:50:37.997421+0000","last_fullsized":"2024-04-26T19:50:37.997421+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130385+0000","last_change":"2024-04-26T19:50:31.105502+0000","last_active":"2024-04-26T19:50:37.130385+0000","last_peered":"2024-04-26T19:50:37.130385+0000","last_clean":"2024-04-26T19:50:37.130385+0000","last_became_active":"2024-04-26T19:50:31.105047+0000","last_became_peered":"2024-04-26T19:50:31.105047+0000","last_unstale":"2024-04-26T19:50:37.130385+0000","last_undegraded":"2024-04-26T19:50:37.130385+0000","last_fullsized":"2024-04-26T19:50:37.130385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:20.874838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653576+0000","last_change":"2024-04-26T19:50:30.110096+0000","last_active":"2024-04-26T19:50:38.653576+0000","last_peered":"2024-04-26T19:50:38.653576+0000","last_clean":"2024-04-26T19:50:38.653576+0000","last_became_active":"2024-04-26T19:50:30.109131+0000","last_became_peered":"2024-04-26T19:50:30.109131+0000","last_unstale":"2024-04-26T19:50:38.653576+0000","last_undegraded":"2024-04-26T19:50:38.653576+0000","last_fullsized":"2024-04-26T19:50:38.653576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997740+0000","last_change":"2024-04-26T19:50:35.148072+0000","last_active":"2024-04-26T19:50:37.997740+0000","last_peered":"2024-04-26T19:50:37.997740+0000","last_clean":"2024-04-26T19:50:37.997740+0000","last_became_active":"2024-04-26T19:50:35.147697+0000","last_became_peered":"2024-04-26T19:50:35.147697+0000","last_unstale":"2024-04-26T19:50:37.997740+0000","last_undegraded":"2024-04-26T19:50:37.997740+0000","last_fullsized":"2024-04-26T19:50:37.997740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997366+0000","last_change":"2024-04-26T19:50:31.108339+0000","last_active":"2024-04-26T19:50:37.997366+0000","last_peered":"2024-04-26T19:50:37.997366+0000","last_clean":"2024-04-26T19:50:37.997366+0000","last_became_active":"2024-04-26T19:50:31.108056+0000","last_became_peered":"2024-04-26T19:50:31.108056+0000","last_unstale":"2024-04-26T19:50:37.997366+0000","last_undegraded":"2024-04-26T19:50:37.997366+0000","last_fullsized":"2024-04-26T19:50:37.997366+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536841+0000","last_change":"2024-04-26T19:50:30.111350+0000","last_active":"2024-04-26T19:50:38.536841+0000","last_peered":"2024-04-26T19:50:38.536841+0000","last_clean":"2024-04-26T19:50:38.536841+0000","last_became_active":"2024-04-26T19:50:30.111070+0000","last_became_peered":"2024-04-26T19:50:30.111070+0000","last_unstale":"2024-04-26T19:50:38.536841+0000","last_undegraded":"2024-04-26T19:50:38.536841+0000","last_fullsized":"2024-04-26T19:50:38.536841+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.998002+0000","last_change":"2024-04-26T19:50:35.146694+0000","last_active":"2024-04-26T19:50:37.998002+0000","last_peered":"2024-04-26T19:50:37.998002+0000","last_clean":"2024-04-26T19:50:37.998002+0000","last_became_active":"2024-04-26T19:50:35.145805+0000","last_became_peered":"2024-04-26T19:50:35.145805+0000","last_unstale":"2024-04-26T19:50:37.998002+0000","last_undegraded":"2024-04-26T19:50:37.998002+0000","last_fullsized":"2024-04-26T19:50:37.998002+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:01.350601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299548+0000","last_change":"2024-04-26T19:50:31.103279+0000","last_active":"2024-04-26T19:50:38.299548+0000","last_peered":"2024-04-26T19:50:38.299548+0000","last_clean":"2024-04-26T19:50:38.299548+0000","last_became_active":"2024-04-26T19:50:31.102858+0000","last_became_peered":"2024-04-26T19:50:31.102858+0000","last_unstale":"2024-04-26T19:50:38.299548+0000","last_undegraded":"2024-04-26T19:50:38.299548+0000","last_fullsized":"2024-04-26T19:50:38.299548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:47:56.497144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653771+0000","last_change":"2024-04-26T19:50:30.109372+0000","last_active":"2024-04-26T19:50:38.653771+0000","last_peered":"2024-04-26T19:50:38.653771+0000","last_clean":"2024-04-26T19:50:38.653771+0000","last_became_active":"2024-04-26T19:50:30.109087+0000","last_became_peered":"2024-04-26T19:50:30.109087+0000","last_unstale":"2024-04-26T19:50:38.653771+0000","last_undegraded":"2024-04-26T19:50:38.653771+0000","last_fullsized":"2024-04-26T19:50:38.653771+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:13.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300021+0000","last_change":"2024-04-26T19:50:35.623867+0000","last_active":"2024-04-26T19:50:38.300021+0000","last_peered":"2024-04-26T19:50:38.300021+0000","last_clean":"2024-04-26T19:50:38.300021+0000","last_became_active":"2024-04-26T19:50:35.623416+0000","last_became_peered":"2024-04-26T19:50:35.623416+0000","last_unstale":"2024-04-26T19:50:38.300021+0000","last_undegraded":"2024-04-26T19:50:38.300021+0000","last_fullsized":"2024-04-26T19:50:38.300021+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:26:02.874670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130394+0000","last_change":"2024-04-26T19:50:31.113014+0000","last_active":"2024-04-26T19:50:37.130394+0000","last_peered":"2024-04-26T19:50:37.130394+0000","last_clean":"2024-04-26T19:50:37.130394+0000","last_became_active":"2024-04-26T19:50:31.112753+0000","last_became_peered":"2024-04-26T19:50:31.112753+0000","last_unstale":"2024-04-26T19:50:37.130394+0000","last_undegraded":"2024-04-26T19:50:37.130394+0000","last_fullsized":"2024-04-26T19:50:37.130394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653084+0000","last_change":"2024-04-26T19:50:30.110312+0000","last_active":"2024-04-26T19:50:38.653084+0000","last_peered":"2024-04-26T19:50:38.653084+0000","last_clean":"2024-04-26T19:50:38.653084+0000","last_became_active":"2024-04-26T19:50:30.109147+0000","last_became_peered":"2024-04-26T19:50:30.109147+0000","last_unstale":"2024-04-26T19:50:38.653084+0000","last_undegraded":"2024-04-26T19:50:38.653084+0000","last_fullsized":"2024-04-26T19:50:38.653084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":180,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654171+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:50:38.654171+0000","last_peered":"2024-04-26T19:50:38.654171+0000","last_clean":"2024-04-26T19:50:38.654171+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:50:38.654171+0000","last_undegraded":"2024-04-26T19:50:38.654171+0000","last_fullsized":"2024-04-26T19:50:38.654171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653046+0000","last_change":"2024-04-26T19:50:35.627345+0000","last_active":"2024-04-26T19:50:38.653046+0000","last_peered":"2024-04-26T19:50:38.653046+0000","last_clean":"2024-04-26T19:50:38.653046+0000","last_became_active":"2024-04-26T19:50:35.626795+0000","last_became_peered":"2024-04-26T19:50:35.626795+0000","last_unstale":"2024-04-26T19:50:38.653046+0000","last_undegraded":"2024-04-26T19:50:38.653046+0000","last_fullsized":"2024-04-26T19:50:38.653046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:14.433127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653270+0000","last_change":"2024-04-26T19:50:31.112950+0000","last_active":"2024-04-26T19:50:38.653270+0000","last_peered":"2024-04-26T19:50:38.653270+0000","last_clean":"2024-04-26T19:50:38.653270+0000","last_became_active":"2024-04-26T19:50:31.112674+0000","last_became_peered":"2024-04-26T19:50:31.112674+0000","last_unstale":"2024-04-26T19:50:38.653270+0000","last_undegraded":"2024-04-26T19:50:38.653270+0000","last_fullsized":"2024-04-26T19:50:38.653270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537253+0000","last_change":"2024-04-26T19:50:30.114819+0000","last_active":"2024-04-26T19:50:38.537253+0000","last_peered":"2024-04-26T19:50:38.537253+0000","last_clean":"2024-04-26T19:50:38.537253+0000","last_became_active":"2024-04-26T19:50:30.114566+0000","last_became_peered":"2024-04-26T19:50:30.114566+0000","last_unstale":"2024-04-26T19:50:38.537253+0000","last_undegraded":"2024-04-26T19:50:38.537253+0000","last_fullsized":"2024-04-26T19:50:38.537253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654387+0000","last_change":"2024-04-26T19:50:35.626488+0000","last_active":"2024-04-26T19:50:38.654387+0000","last_peered":"2024-04-26T19:50:38.654387+0000","last_clean":"2024-04-26T19:50:38.654387+0000","last_became_active":"2024-04-26T19:50:35.625951+0000","last_became_peered":"2024-04-26T19:50:35.625951+0000","last_unstale":"2024-04-26T19:50:38.654387+0000","last_undegraded":"2024-04-26T19:50:38.654387+0000","last_fullsized":"2024-04-26T19:50:38.654387+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:46.591736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652684+0000","last_change":"2024-04-26T19:50:31.119380+0000","last_active":"2024-04-26T19:50:38.652684+0000","last_peered":"2024-04-26T19:50:38.652684+0000","last_clean":"2024-04-26T19:50:38.652684+0000","last_became_active":"2024-04-26T19:50:31.118011+0000","last_became_peered":"2024-04-26T19:50:31.118011+0000","last_unstale":"2024-04-26T19:50:38.652684+0000","last_undegraded":"2024-04-26T19:50:38.652684+0000","last_fullsized":"2024-04-26T19:50:38.652684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:13.577254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997787+0000","last_change":"2024-04-26T19:50:30.101571+0000","last_active":"2024-04-26T19:50:37.997787+0000","last_peered":"2024-04-26T19:50:37.997787+0000","last_clean":"2024-04-26T19:50:37.997787+0000","last_became_active":"2024-04-26T19:50:30.101257+0000","last_became_peered":"2024-04-26T19:50:30.101257+0000","last_unstale":"2024-04-26T19:50:37.997787+0000","last_undegraded":"2024-04-26T19:50:37.997787+0000","last_fullsized":"2024-04-26T19:50:37.997787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:13.974823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299968+0000","last_change":"2024-04-26T19:50:35.146304+0000","last_active":"2024-04-26T19:50:38.299968+0000","last_peered":"2024-04-26T19:50:38.299968+0000","last_clean":"2024-04-26T19:50:38.299968+0000","last_became_active":"2024-04-26T19:50:35.145918+0000","last_became_peered":"2024-04-26T19:50:35.145918+0000","last_unstale":"2024-04-26T19:50:38.299968+0000","last_undegraded":"2024-04-26T19:50:38.299968+0000","last_fullsized":"2024-04-26T19:50:38.299968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:05:01.847505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653338+0000","last_change":"2024-04-26T19:50:31.115622+0000","last_active":"2024-04-26T19:50:38.653338+0000","last_peered":"2024-04-26T19:50:38.653338+0000","last_clean":"2024-04-26T19:50:38.653338+0000","last_became_active":"2024-04-26T19:50:31.115358+0000","last_became_peered":"2024-04-26T19:50:31.115358+0000","last_unstale":"2024-04-26T19:50:38.653338+0000","last_undegraded":"2024-04-26T19:50:38.653338+0000","last_fullsized":"2024-04-26T19:50:38.653338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:31.186700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536869+0000","last_change":"2024-04-26T19:50:30.112087+0000","last_active":"2024-04-26T19:50:38.536869+0000","last_peered":"2024-04-26T19:50:38.536869+0000","last_clean":"2024-04-26T19:50:38.536869+0000","last_became_active":"2024-04-26T19:50:30.111773+0000","last_became_peered":"2024-04-26T19:50:30.111773+0000","last_unstale":"2024-04-26T19:50:38.536869+0000","last_undegraded":"2024-04-26T19:50:38.536869+0000","last_fullsized":"2024-04-26T19:50:38.536869+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652160+0000","last_change":"2024-04-26T19:50:35.624170+0000","last_active":"2024-04-26T19:50:38.652160+0000","last_peered":"2024-04-26T19:50:38.652160+0000","last_clean":"2024-04-26T19:50:38.652160+0000","last_became_active":"2024-04-26T19:50:35.623758+0000","last_became_peered":"2024-04-26T19:50:35.623758+0000","last_unstale":"2024-04-26T19:50:38.652160+0000","last_undegraded":"2024-04-26T19:50:38.652160+0000","last_fullsized":"2024-04-26T19:50:38.652160+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:01.350601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653614+0000","last_change":"2024-04-26T19:50:31.115100+0000","last_active":"2024-04-26T19:50:38.653614+0000","last_peered":"2024-04-26T19:50:38.653614+0000","last_clean":"2024-04-26T19:50:38.653614+0000","last_became_active":"2024-04-26T19:50:31.114558+0000","last_became_peered":"2024-04-26T19:50:31.114558+0000","last_unstale":"2024-04-26T19:50:38.653614+0000","last_undegraded":"2024-04-26T19:50:38.653614+0000","last_fullsized":"2024-04-26T19:50:38.653614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:42.558104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130407+0000","last_change":"2024-04-26T19:50:30.100580+0000","last_active":"2024-04-26T19:50:37.130407+0000","last_peered":"2024-04-26T19:50:37.130407+0000","last_clean":"2024-04-26T19:50:37.130407+0000","last_became_active":"2024-04-26T19:50:30.099562+0000","last_became_peered":"2024-04-26T19:50:30.099562+0000","last_unstale":"2024-04-26T19:50:37.130407+0000","last_undegraded":"2024-04-26T19:50:37.130407+0000","last_fullsized":"2024-04-26T19:50:37.130407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:50:02.207190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129529+0000","last_change":"2024-04-26T19:50:35.143446+0000","last_active":"2024-04-26T19:50:37.129529+0000","last_peered":"2024-04-26T19:50:37.129529+0000","last_clean":"2024-04-26T19:50:37.129529+0000","last_became_active":"2024-04-26T19:50:35.143028+0000","last_became_peered":"2024-04-26T19:50:35.143028+0000","last_unstale":"2024-04-26T19:50:37.129529+0000","last_undegraded":"2024-04-26T19:50:37.129529+0000","last_fullsized":"2024-04-26T19:50:37.129529+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:43.974896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653088+0000","last_change":"2024-04-26T19:50:31.116546+0000","last_active":"2024-04-26T19:50:38.653088+0000","last_peered":"2024-04-26T19:50:38.653088+0000","last_clean":"2024-04-26T19:50:38.653088+0000","last_became_active":"2024-04-26T19:50:31.115261+0000","last_became_peered":"2024-04-26T19:50:31.115261+0000","last_unstale":"2024-04-26T19:50:38.653088+0000","last_undegraded":"2024-04-26T19:50:38.653088+0000","last_fullsized":"2024-04-26T19:50:38.653088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.212877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536837+0000","last_change":"2024-04-26T19:50:30.099768+0000","last_active":"2024-04-26T19:50:38.536837+0000","last_peered":"2024-04-26T19:50:38.536837+0000","last_clean":"2024-04-26T19:50:38.536837+0000","last_became_active":"2024-04-26T19:50:30.099471+0000","last_became_peered":"2024-04-26T19:50:30.099471+0000","last_unstale":"2024-04-26T19:50:38.536837+0000","last_undegraded":"2024-04-26T19:50:38.536837+0000","last_fullsized":"2024-04-26T19:50:38.536837+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299923+0000","last_change":"2024-04-26T19:50:35.148077+0000","last_active":"2024-04-26T19:50:38.299923+0000","last_peered":"2024-04-26T19:50:38.299923+0000","last_clean":"2024-04-26T19:50:38.299923+0000","last_became_active":"2024-04-26T19:50:35.147552+0000","last_became_peered":"2024-04-26T19:50:35.147552+0000","last_unstale":"2024-04-26T19:50:38.299923+0000","last_undegraded":"2024-04-26T19:50:38.299923+0000","last_fullsized":"2024-04-26T19:50:38.299923+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:03:11.052818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652600+0000","last_change":"2024-04-26T19:50:31.105398+0000","last_active":"2024-04-26T19:50:38.652600+0000","last_peered":"2024-04-26T19:50:38.652600+0000","last_clean":"2024-04-26T19:50:38.652600+0000","last_became_active":"2024-04-26T19:50:31.105140+0000","last_became_peered":"2024-04-26T19:50:31.105140+0000","last_unstale":"2024-04-26T19:50:38.652600+0000","last_undegraded":"2024-04-26T19:50:38.652600+0000","last_fullsized":"2024-04-26T19:50:38.652600+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.651966+0000","last_change":"2024-04-26T19:50:30.111019+0000","last_active":"2024-04-26T19:50:38.651966+0000","last_peered":"2024-04-26T19:50:38.651966+0000","last_clean":"2024-04-26T19:50:38.651966+0000","last_became_active":"2024-04-26T19:50:30.110590+0000","last_became_peered":"2024-04-26T19:50:30.110590+0000","last_unstale":"2024-04-26T19:50:38.651966+0000","last_undegraded":"2024-04-26T19:50:38.651966+0000","last_fullsized":"2024-04-26T19:50:38.651966+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.131001+0000","last_change":"2024-04-26T19:50:35.625721+0000","last_active":"2024-04-26T19:50:37.131001+0000","last_peered":"2024-04-26T19:50:37.131001+0000","last_clean":"2024-04-26T19:50:37.131001+0000","last_became_active":"2024-04-26T19:50:35.625294+0000","last_became_peered":"2024-04-26T19:50:35.625294+0000","last_unstale":"2024-04-26T19:50:37.131001+0000","last_undegraded":"2024-04-26T19:50:37.131001+0000","last_fullsized":"2024-04-26T19:50:37.131001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:57:18.158960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653765+0000","last_change":"2024-04-26T19:50:31.113984+0000","last_active":"2024-04-26T19:50:38.653765+0000","last_peered":"2024-04-26T19:50:38.653765+0000","last_clean":"2024-04-26T19:50:38.653765+0000","last_became_active":"2024-04-26T19:50:31.113723+0000","last_became_peered":"2024-04-26T19:50:31.113723+0000","last_unstale":"2024-04-26T19:50:38.653765+0000","last_undegraded":"2024-04-26T19:50:38.653765+0000","last_fullsized":"2024-04-26T19:50:38.653765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:54:25.465290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653698+0000","last_change":"2024-04-26T19:50:30.110687+0000","last_active":"2024-04-26T19:50:38.653698+0000","last_peered":"2024-04-26T19:50:38.653698+0000","last_clean":"2024-04-26T19:50:38.653698+0000","last_became_active":"2024-04-26T19:50:30.110423+0000","last_became_peered":"2024-04-26T19:50:30.110423+0000","last_unstale":"2024-04-26T19:50:38.653698+0000","last_undegraded":"2024-04-26T19:50:38.653698+0000","last_fullsized":"2024-04-26T19:50:38.653698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:21:12.135200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130741+0000","last_change":"2024-04-26T19:50:35.623855+0000","last_active":"2024-04-26T19:50:37.130741+0000","last_peered":"2024-04-26T19:50:37.130741+0000","last_clean":"2024-04-26T19:50:37.130741+0000","last_became_active":"2024-04-26T19:50:35.623453+0000","last_became_peered":"2024-04-26T19:50:35.623453+0000","last_unstale":"2024-04-26T19:50:37.130741+0000","last_undegraded":"2024-04-26T19:50:37.130741+0000","last_fullsized":"2024-04-26T19:50:37.130741+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:34:21.199810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299726+0000","last_change":"2024-04-26T19:50:31.114891+0000","last_active":"2024-04-26T19:50:38.299726+0000","last_peered":"2024-04-26T19:50:38.299726+0000","last_clean":"2024-04-26T19:50:38.299726+0000","last_became_active":"2024-04-26T19:50:31.114338+0000","last_became_peered":"2024-04-26T19:50:31.114338+0000","last_unstale":"2024-04-26T19:50:38.299726+0000","last_undegraded":"2024-04-26T19:50:38.299726+0000","last_fullsized":"2024-04-26T19:50:38.299726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299301+0000","last_change":"2024-04-26T19:50:30.108408+0000","last_active":"2024-04-26T19:50:38.299301+0000","last_peered":"2024-04-26T19:50:38.299301+0000","last_clean":"2024-04-26T19:50:38.299301+0000","last_became_active":"2024-04-26T19:50:30.107261+0000","last_became_peered":"2024-04-26T19:50:30.107261+0000","last_unstale":"2024-04-26T19:50:38.299301+0000","last_undegraded":"2024-04-26T19:50:38.299301+0000","last_fullsized":"2024-04-26T19:50:38.299301+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:21.153340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654534+0000","last_change":"2024-04-26T19:50:35.624403+0000","last_active":"2024-04-26T19:50:38.654534+0000","last_peered":"2024-04-26T19:50:38.654534+0000","last_clean":"2024-04-26T19:50:38.654534+0000","last_became_active":"2024-04-26T19:50:35.623811+0000","last_became_peered":"2024-04-26T19:50:35.623811+0000","last_unstale":"2024-04-26T19:50:38.654534+0000","last_undegraded":"2024-04-26T19:50:38.654534+0000","last_fullsized":"2024-04-26T19:50:38.654534+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:11:46.747785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653171+0000","last_change":"2024-04-26T19:50:31.115452+0000","last_active":"2024-04-26T19:50:38.653171+0000","last_peered":"2024-04-26T19:50:38.653171+0000","last_clean":"2024-04-26T19:50:38.653171+0000","last_became_active":"2024-04-26T19:50:31.114780+0000","last_became_peered":"2024-04-26T19:50:31.114780+0000","last_unstale":"2024-04-26T19:50:38.653171+0000","last_undegraded":"2024-04-26T19:50:38.653171+0000","last_fullsized":"2024-04-26T19:50:38.653171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:40.502106+0000","last_change":"2024-04-26T19:50:30.110540+0000","last_active":"2024-04-26T19:50:40.502106+0000","last_peered":"2024-04-26T19:50:40.502106+0000","last_clean":"2024-04-26T19:50:40.502106+0000","last_became_active":"2024-04-26T19:50:30.110241+0000","last_became_peered":"2024-04-26T19:50:30.110241+0000","last_unstale":"2024-04-26T19:50:40.502106+0000","last_undegraded":"2024-04-26T19:50:40.502106+0000","last_fullsized":"2024-04-26T19:50:40.502106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997615+0000","last_change":"2024-04-26T19:50:35.149139+0000","last_active":"2024-04-26T19:50:37.997615+0000","last_peered":"2024-04-26T19:50:37.997615+0000","last_clean":"2024-04-26T19:50:37.997615+0000","last_became_active":"2024-04-26T19:50:35.148663+0000","last_became_peered":"2024-04-26T19:50:35.148663+0000","last_unstale":"2024-04-26T19:50:37.997615+0000","last_undegraded":"2024-04-26T19:50:37.997615+0000","last_fullsized":"2024-04-26T19:50:37.997615+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130224+0000","last_change":"2024-04-26T19:50:31.112921+0000","last_active":"2024-04-26T19:50:37.130224+0000","last_peered":"2024-04-26T19:50:37.130224+0000","last_clean":"2024-04-26T19:50:37.130224+0000","last_became_active":"2024-04-26T19:50:31.112651+0000","last_became_peered":"2024-04-26T19:50:31.112651+0000","last_unstale":"2024-04-26T19:50:37.130224+0000","last_undegraded":"2024-04-26T19:50:37.130224+0000","last_fullsized":"2024-04-26T19:50:37.130224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:13.149763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997777+0000","last_change":"2024-04-26T19:50:30.110464+0000","last_active":"2024-04-26T19:50:37.997777+0000","last_peered":"2024-04-26T19:50:37.997777+0000","last_clean":"2024-04-26T19:50:37.997777+0000","last_became_active":"2024-04-26T19:50:30.110101+0000","last_became_peered":"2024-04-26T19:50:30.110101+0000","last_unstale":"2024-04-26T19:50:37.997777+0000","last_undegraded":"2024-04-26T19:50:37.997777+0000","last_fullsized":"2024-04-26T19:50:37.997777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:12.144175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130067+0000","last_change":"2024-04-26T19:50:35.143854+0000","last_active":"2024-04-26T19:50:37.130067+0000","last_peered":"2024-04-26T19:50:37.130067+0000","last_clean":"2024-04-26T19:50:37.130067+0000","last_became_active":"2024-04-26T19:50:35.143264+0000","last_became_peered":"2024-04-26T19:50:35.143264+0000","last_unstale":"2024-04-26T19:50:37.130067+0000","last_undegraded":"2024-04-26T19:50:37.130067+0000","last_fullsized":"2024-04-26T19:50:37.130067+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:43:10.922204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300209+0000","last_change":"2024-04-26T19:50:31.115638+0000","last_active":"2024-04-26T19:50:38.300209+0000","last_peered":"2024-04-26T19:50:38.300209+0000","last_clean":"2024-04-26T19:50:38.300209+0000","last_became_active":"2024-04-26T19:50:31.113889+0000","last_became_peered":"2024-04-26T19:50:31.113889+0000","last_unstale":"2024-04-26T19:50:38.300209+0000","last_undegraded":"2024-04-26T19:50:38.300209+0000","last_fullsized":"2024-04-26T19:50:38.300209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652958+0000","last_change":"2024-04-26T19:50:30.109848+0000","last_active":"2024-04-26T19:50:38.652958+0000","last_peered":"2024-04-26T19:50:38.652958+0000","last_clean":"2024-04-26T19:50:38.652958+0000","last_became_active":"2024-04-26T19:50:30.109407+0000","last_became_peered":"2024-04-26T19:50:30.109407+0000","last_unstale":"2024-04-26T19:50:38.652958+0000","last_undegraded":"2024-04-26T19:50:38.652958+0000","last_fullsized":"2024-04-26T19:50:38.652958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130343+0000","last_change":"2024-04-26T19:50:35.143221+0000","last_active":"2024-04-26T19:50:37.130343+0000","last_peered":"2024-04-26T19:50:37.130343+0000","last_clean":"2024-04-26T19:50:37.130343+0000","last_became_active":"2024-04-26T19:50:35.142868+0000","last_became_peered":"2024-04-26T19:50:35.142868+0000","last_unstale":"2024-04-26T19:50:37.130343+0000","last_undegraded":"2024-04-26T19:50:37.130343+0000","last_fullsized":"2024-04-26T19:50:37.130343+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:37:16.097115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652135+0000","last_change":"2024-04-26T19:50:31.115770+0000","last_active":"2024-04-26T19:50:38.652135+0000","last_peered":"2024-04-26T19:50:38.652135+0000","last_clean":"2024-04-26T19:50:38.652135+0000","last_became_active":"2024-04-26T19:50:31.115462+0000","last_became_peered":"2024-04-26T19:50:31.115462+0000","last_unstale":"2024-04-26T19:50:38.652135+0000","last_undegraded":"2024-04-26T19:50:38.652135+0000","last_fullsized":"2024-04-26T19:50:38.652135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:23:14.212877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653880+0000","last_change":"2024-04-26T19:50:30.110213+0000","last_active":"2024-04-26T19:50:38.653880+0000","last_peered":"2024-04-26T19:50:38.653880+0000","last_clean":"2024-04-26T19:50:38.653880+0000","last_became_active":"2024-04-26T19:50:30.109923+0000","last_became_peered":"2024-04-26T19:50:30.109923+0000","last_unstale":"2024-04-26T19:50:38.653880+0000","last_undegraded":"2024-04-26T19:50:38.653880+0000","last_fullsized":"2024-04-26T19:50:38.653880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:52:19.869250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130324+0000","last_change":"2024-04-26T19:50:35.624637+0000","last_active":"2024-04-26T19:50:37.130324+0000","last_peered":"2024-04-26T19:50:37.130324+0000","last_clean":"2024-04-26T19:50:37.130324+0000","last_became_active":"2024-04-26T19:50:35.624195+0000","last_became_peered":"2024-04-26T19:50:35.624195+0000","last_unstale":"2024-04-26T19:50:37.130324+0000","last_undegraded":"2024-04-26T19:50:37.130324+0000","last_fullsized":"2024-04-26T19:50:37.130324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:29:16.049053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299925+0000","last_change":"2024-04-26T19:50:35.626111+0000","last_active":"2024-04-26T19:50:38.299925+0000","last_peered":"2024-04-26T19:50:38.299925+0000","last_clean":"2024-04-26T19:50:38.299925+0000","last_became_active":"2024-04-26T19:50:35.625678+0000","last_became_peered":"2024-04-26T19:50:35.625678+0000","last_unstale":"2024-04-26T19:50:38.299925+0000","last_undegraded":"2024-04-26T19:50:38.299925+0000","last_fullsized":"2024-04-26T19:50:38.299925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:30:18.280989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997291+0000","last_change":"2024-04-26T19:50:31.108163+0000","last_active":"2024-04-26T19:50:37.997291+0000","last_peered":"2024-04-26T19:50:37.997291+0000","last_clean":"2024-04-26T19:50:37.997291+0000","last_became_active":"2024-04-26T19:50:31.107840+0000","last_became_peered":"2024-04-26T19:50:31.107840+0000","last_unstale":"2024-04-26T19:50:37.997291+0000","last_undegraded":"2024-04-26T19:50:37.997291+0000","last_fullsized":"2024-04-26T19:50:37.997291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.651925+0000","last_change":"2024-04-26T19:50:30.111205+0000","last_active":"2024-04-26T19:50:38.651925+0000","last_peered":"2024-04-26T19:50:38.651925+0000","last_clean":"2024-04-26T19:50:38.651925+0000","last_became_active":"2024-04-26T19:50:30.110491+0000","last_became_peered":"2024-04-26T19:50:30.110491+0000","last_unstale":"2024-04-26T19:50:38.651925+0000","last_undegraded":"2024-04-26T19:50:38.651925+0000","last_fullsized":"2024-04-26T19:50:38.651925+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:53.405030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653880+0000","last_change":"2024-04-26T19:50:35.144010+0000","last_active":"2024-04-26T19:50:38.653880+0000","last_peered":"2024-04-26T19:50:38.653880+0000","last_clean":"2024-04-26T19:50:38.653880+0000","last_became_active":"2024-04-26T19:50:35.143558+0000","last_became_peered":"2024-04-26T19:50:35.143558+0000","last_unstale":"2024-04-26T19:50:38.653880+0000","last_undegraded":"2024-04-26T19:50:38.653880+0000","last_fullsized":"2024-04-26T19:50:38.653880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:10.325286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130338+0000","last_change":"2024-04-26T19:50:31.106592+0000","last_active":"2024-04-26T19:50:37.130338+0000","last_peered":"2024-04-26T19:50:37.130338+0000","last_clean":"2024-04-26T19:50:37.130338+0000","last_became_active":"2024-04-26T19:50:31.106298+0000","last_became_peered":"2024-04-26T19:50:31.106298+0000","last_unstale":"2024-04-26T19:50:37.130338+0000","last_undegraded":"2024-04-26T19:50:37.130338+0000","last_fullsized":"2024-04-26T19:50:37.130338+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130435+0000","last_change":"2024-04-26T19:50:30.110713+0000","last_active":"2024-04-26T19:50:37.130435+0000","last_peered":"2024-04-26T19:50:37.130435+0000","last_clean":"2024-04-26T19:50:37.130435+0000","last_became_active":"2024-04-26T19:50:30.109274+0000","last_became_peered":"2024-04-26T19:50:30.109274+0000","last_unstale":"2024-04-26T19:50:37.130435+0000","last_undegraded":"2024-04-26T19:50:37.130435+0000","last_fullsized":"2024-04-26T19:50:37.130435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:10:49.861748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654164+0000","last_change":"2024-04-26T19:50:30.111443+0000","last_active":"2024-04-26T19:50:38.654164+0000","last_peered":"2024-04-26T19:50:38.654164+0000","last_clean":"2024-04-26T19:50:38.654164+0000","last_became_active":"2024-04-26T19:50:30.111159+0000","last_became_peered":"2024-04-26T19:50:30.111159+0000","last_unstale":"2024-04-26T19:50:38.654164+0000","last_undegraded":"2024-04-26T19:50:38.654164+0000","last_fullsized":"2024-04-26T19:50:38.654164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:23:29.802076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300079+0000","last_change":"2024-04-26T19:50:31.104770+0000","last_active":"2024-04-26T19:50:38.300079+0000","last_peered":"2024-04-26T19:50:38.300079+0000","last_clean":"2024-04-26T19:50:38.300079+0000","last_became_active":"2024-04-26T19:50:31.104100+0000","last_became_peered":"2024-04-26T19:50:31.104100+0000","last_unstale":"2024-04-26T19:50:38.300079+0000","last_undegraded":"2024-04-26T19:50:38.300079+0000","last_fullsized":"2024-04-26T19:50:38.300079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537176+0000","last_change":"2024-04-26T19:50:35.143179+0000","last_active":"2024-04-26T19:50:38.537176+0000","last_peered":"2024-04-26T19:50:38.537176+0000","last_clean":"2024-04-26T19:50:38.537176+0000","last_became_active":"2024-04-26T19:50:35.142761+0000","last_became_peered":"2024-04-26T19:50:35.142761+0000","last_unstale":"2024-04-26T19:50:38.537176+0000","last_undegraded":"2024-04-26T19:50:38.537176+0000","last_fullsized":"2024-04-26T19:50:38.537176+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:42.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130613+0000","last_change":"2024-04-26T19:50:30.115508+0000","last_active":"2024-04-26T19:50:37.130613+0000","last_peered":"2024-04-26T19:50:37.130613+0000","last_clean":"2024-04-26T19:50:37.130613+0000","last_became_active":"2024-04-26T19:50:30.115235+0000","last_became_peered":"2024-04-26T19:50:30.115235+0000","last_unstale":"2024-04-26T19:50:37.130613+0000","last_undegraded":"2024-04-26T19:50:37.130613+0000","last_fullsized":"2024-04-26T19:50:37.130613+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:29.074543+0000","last_clean_scrub_stamp":"2024-04-26T19:50:29.074543+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:59:31.691383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.129914+0000","last_change":"2024-04-26T19:50:31.110879+0000","last_active":"2024-04-26T19:50:37.129914+0000","last_peered":"2024-04-26T19:50:37.129914+0000","last_clean":"2024-04-26T19:50:37.129914+0000","last_became_active":"2024-04-26T19:50:31.110592+0000","last_became_peered":"2024-04-26T19:50:31.110592+0000","last_unstale":"2024-04-26T19:50:37.129914+0000","last_undegraded":"2024-04-26T19:50:37.129914+0000","last_fullsized":"2024-04-26T19:50:37.129914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:58:54.410618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130887+0000","last_change":"2024-04-26T19:50:35.144257+0000","last_active":"2024-04-26T19:50:37.130887+0000","last_peered":"2024-04-26T19:50:37.130887+0000","last_clean":"2024-04-26T19:50:37.130887+0000","last_became_active":"2024-04-26T19:50:35.143853+0000","last_became_peered":"2024-04-26T19:50:35.143853+0000","last_unstale":"2024-04-26T19:50:37.130887+0000","last_undegraded":"2024-04-26T19:50:37.130887+0000","last_fullsized":"2024-04-26T19:50:37.130887+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:34.113763+0000","last_clean_scrub_stamp":"2024-04-26T19:50:34.113763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:14:05.350339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652235+0000","last_change":"2024-04-26T19:50:31.105355+0000","last_active":"2024-04-26T19:50:38.652235+0000","last_peered":"2024-04-26T19:50:38.652235+0000","last_clean":"2024-04-26T19:50:38.652235+0000","last_became_active":"2024-04-26T19:50:31.105085+0000","last_became_peered":"2024-04-26T19:50:31.105085+0000","last_unstale":"2024-04-26T19:50:38.652235+0000","last_undegraded":"2024-04-26T19:50:38.652235+0000","last_fullsized":"2024-04-26T19:50:38.652235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.536962+0000","last_change":"2024-04-26T19:50:31.112415+0000","last_active":"2024-04-26T19:50:38.536962+0000","last_peered":"2024-04-26T19:50:38.536962+0000","last_clean":"2024-04-26T19:50:38.536962+0000","last_became_active":"2024-04-26T19:50:31.112126+0000","last_became_peered":"2024-04-26T19:50:31.112126+0000","last_unstale":"2024-04-26T19:50:38.536962+0000","last_undegraded":"2024-04-26T19:50:38.536962+0000","last_fullsized":"2024-04-26T19:50:38.536962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:21:13.149763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654244+0000","last_change":"2024-04-26T19:50:31.121580+0000","last_active":"2024-04-26T19:50:38.654244+0000","last_peered":"2024-04-26T19:50:38.654244+0000","last_clean":"2024-04-26T19:50:38.654244+0000","last_became_active":"2024-04-26T19:50:31.121296+0000","last_became_peered":"2024-04-26T19:50:31.121296+0000","last_unstale":"2024-04-26T19:50:38.654244+0000","last_undegraded":"2024-04-26T19:50:38.654244+0000","last_fullsized":"2024-04-26T19:50:38.654244+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.291653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.300008+0000","last_change":"2024-04-26T19:50:31.106600+0000","last_active":"2024-04-26T19:50:38.300008+0000","last_peered":"2024-04-26T19:50:38.300008+0000","last_clean":"2024-04-26T19:50:38.300008+0000","last_became_active":"2024-04-26T19:50:31.106120+0000","last_became_peered":"2024-04-26T19:50:31.106120+0000","last_unstale":"2024-04-26T19:50:38.300008+0000","last_undegraded":"2024-04-26T19:50:38.300008+0000","last_fullsized":"2024-04-26T19:50:38.300008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130813+0000","last_change":"2024-04-26T19:50:31.113865+0000","last_active":"2024-04-26T19:50:37.130813+0000","last_peered":"2024-04-26T19:50:37.130813+0000","last_clean":"2024-04-26T19:50:37.130813+0000","last_became_active":"2024-04-26T19:50:31.113603+0000","last_became_peered":"2024-04-26T19:50:31.113603+0000","last_unstale":"2024-04-26T19:50:37.130813+0000","last_undegraded":"2024-04-26T19:50:37.130813+0000","last_fullsized":"2024-04-26T19:50:37.130813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:21.810799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997661+0000","last_change":"2024-04-26T19:50:31.113048+0000","last_active":"2024-04-26T19:50:37.997661+0000","last_peered":"2024-04-26T19:50:37.997661+0000","last_clean":"2024-04-26T19:50:37.997661+0000","last_became_active":"2024-04-26T19:50:31.112760+0000","last_became_peered":"2024-04-26T19:50:31.112760+0000","last_unstale":"2024-04-26T19:50:37.997661+0000","last_undegraded":"2024-04-26T19:50:37.997661+0000","last_fullsized":"2024-04-26T19:50:37.997661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653673+0000","last_change":"2024-04-26T19:50:31.116289+0000","last_active":"2024-04-26T19:50:38.653673+0000","last_peered":"2024-04-26T19:50:38.653673+0000","last_clean":"2024-04-26T19:50:38.653673+0000","last_became_active":"2024-04-26T19:50:31.116030+0000","last_became_peered":"2024-04-26T19:50:31.116030+0000","last_unstale":"2024-04-26T19:50:38.653673+0000","last_undegraded":"2024-04-26T19:50:38.653673+0000","last_fullsized":"2024-04-26T19:50:38.653673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130946+0000","last_change":"2024-04-26T19:50:31.106460+0000","last_active":"2024-04-26T19:50:37.130946+0000","last_peered":"2024-04-26T19:50:37.130946+0000","last_clean":"2024-04-26T19:50:37.130946+0000","last_became_active":"2024-04-26T19:50:31.106066+0000","last_became_peered":"2024-04-26T19:50:31.106066+0000","last_unstale":"2024-04-26T19:50:37.130946+0000","last_undegraded":"2024-04-26T19:50:37.130946+0000","last_fullsized":"2024-04-26T19:50:37.130946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652449+0000","last_change":"2024-04-26T19:50:31.121061+0000","last_active":"2024-04-26T19:50:38.652449+0000","last_peered":"2024-04-26T19:50:38.652449+0000","last_clean":"2024-04-26T19:50:38.652449+0000","last_became_active":"2024-04-26T19:50:31.120795+0000","last_became_peered":"2024-04-26T19:50:31.120795+0000","last_unstale":"2024-04-26T19:50:38.652449+0000","last_undegraded":"2024-04-26T19:50:38.652449+0000","last_fullsized":"2024-04-26T19:50:38.652449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130353+0000","last_change":"2024-04-26T19:50:31.108437+0000","last_active":"2024-04-26T19:50:37.130353+0000","last_peered":"2024-04-26T19:50:37.130353+0000","last_clean":"2024-04-26T19:50:37.130353+0000","last_became_active":"2024-04-26T19:50:31.108163+0000","last_became_peered":"2024-04-26T19:50:31.108163+0000","last_unstale":"2024-04-26T19:50:37.130353+0000","last_undegraded":"2024-04-26T19:50:37.130353+0000","last_fullsized":"2024-04-26T19:50:37.130353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:54:09.224781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654157+0000","last_change":"2024-04-26T19:50:31.117685+0000","last_active":"2024-04-26T19:50:38.654157+0000","last_peered":"2024-04-26T19:50:38.654157+0000","last_clean":"2024-04-26T19:50:38.654157+0000","last_became_active":"2024-04-26T19:50:31.115588+0000","last_became_peered":"2024-04-26T19:50:31.115588+0000","last_unstale":"2024-04-26T19:50:38.654157+0000","last_undegraded":"2024-04-26T19:50:38.654157+0000","last_fullsized":"2024-04-26T19:50:38.654157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:10.399495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299776+0000","last_change":"2024-04-26T19:50:31.115273+0000","last_active":"2024-04-26T19:50:38.299776+0000","last_peered":"2024-04-26T19:50:38.299776+0000","last_clean":"2024-04-26T19:50:38.299776+0000","last_became_active":"2024-04-26T19:50:31.114772+0000","last_became_peered":"2024-04-26T19:50:31.114772+0000","last_unstale":"2024-04-26T19:50:38.299776+0000","last_undegraded":"2024-04-26T19:50:38.299776+0000","last_fullsized":"2024-04-26T19:50:38.299776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653493+0000","last_change":"2024-04-26T19:50:31.110492+0000","last_active":"2024-04-26T19:50:38.653493+0000","last_peered":"2024-04-26T19:50:38.653493+0000","last_clean":"2024-04-26T19:50:38.653493+0000","last_became_active":"2024-04-26T19:50:31.110280+0000","last_became_peered":"2024-04-26T19:50:31.110280+0000","last_unstale":"2024-04-26T19:50:38.653493+0000","last_undegraded":"2024-04-26T19:50:38.653493+0000","last_fullsized":"2024-04-26T19:50:38.653493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653886+0000","last_change":"2024-04-26T19:50:31.114136+0000","last_active":"2024-04-26T19:50:38.653886+0000","last_peered":"2024-04-26T19:50:38.653886+0000","last_clean":"2024-04-26T19:50:38.653886+0000","last_became_active":"2024-04-26T19:50:31.113175+0000","last_became_peered":"2024-04-26T19:50:31.113175+0000","last_unstale":"2024-04-26T19:50:38.653886+0000","last_undegraded":"2024-04-26T19:50:38.653886+0000","last_fullsized":"2024-04-26T19:50:38.653886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:39.941264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.537153+0000","last_change":"2024-04-26T19:50:31.108339+0000","last_active":"2024-04-26T19:50:38.537153+0000","last_peered":"2024-04-26T19:50:38.537153+0000","last_clean":"2024-04-26T19:50:38.537153+0000","last_became_active":"2024-04-26T19:50:31.108092+0000","last_became_peered":"2024-04-26T19:50:31.108092+0000","last_unstale":"2024-04-26T19:50:38.537153+0000","last_undegraded":"2024-04-26T19:50:38.537153+0000","last_fullsized":"2024-04-26T19:50:38.537153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:14.980411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653311+0000","last_change":"2024-04-26T19:50:31.116207+0000","last_active":"2024-04-26T19:50:38.653311+0000","last_peered":"2024-04-26T19:50:38.653311+0000","last_clean":"2024-04-26T19:50:38.653311+0000","last_became_active":"2024-04-26T19:50:31.115498+0000","last_became_peered":"2024-04-26T19:50:31.115498+0000","last_unstale":"2024-04-26T19:50:38.653311+0000","last_undegraded":"2024-04-26T19:50:38.653311+0000","last_fullsized":"2024-04-26T19:50:38.653311+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:34:42.558104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652346+0000","last_change":"2024-04-26T19:50:31.117352+0000","last_active":"2024-04-26T19:50:38.652346+0000","last_peered":"2024-04-26T19:50:38.652346+0000","last_clean":"2024-04-26T19:50:38.652346+0000","last_became_active":"2024-04-26T19:50:31.117098+0000","last_became_peered":"2024-04-26T19:50:31.117098+0000","last_unstale":"2024-04-26T19:50:38.652346+0000","last_undegraded":"2024-04-26T19:50:38.652346+0000","last_fullsized":"2024-04-26T19:50:38.652346+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997376+0000","last_change":"2024-04-26T19:50:31.113884+0000","last_active":"2024-04-26T19:50:37.997376+0000","last_peered":"2024-04-26T19:50:37.997376+0000","last_clean":"2024-04-26T19:50:37.997376+0000","last_became_active":"2024-04-26T19:50:31.113607+0000","last_became_peered":"2024-04-26T19:50:31.113607+0000","last_unstale":"2024-04-26T19:50:37.997376+0000","last_undegraded":"2024-04-26T19:50:37.997376+0000","last_fullsized":"2024-04-26T19:50:37.997376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:10:32.141976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299710+0000","last_change":"2024-04-26T19:50:31.110086+0000","last_active":"2024-04-26T19:50:38.299710+0000","last_peered":"2024-04-26T19:50:38.299710+0000","last_clean":"2024-04-26T19:50:38.299710+0000","last_became_active":"2024-04-26T19:50:31.109800+0000","last_became_peered":"2024-04-26T19:50:31.109800+0000","last_unstale":"2024-04-26T19:50:38.299710+0000","last_undegraded":"2024-04-26T19:50:38.299710+0000","last_fullsized":"2024-04-26T19:50:38.299710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:45.295846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130514+0000","last_change":"2024-04-26T19:50:31.109078+0000","last_active":"2024-04-26T19:50:37.130514+0000","last_peered":"2024-04-26T19:50:37.130514+0000","last_clean":"2024-04-26T19:50:37.130514+0000","last_became_active":"2024-04-26T19:50:31.107801+0000","last_became_peered":"2024-04-26T19:50:31.107801+0000","last_unstale":"2024-04-26T19:50:37.130514+0000","last_undegraded":"2024-04-26T19:50:37.130514+0000","last_fullsized":"2024-04-26T19:50:37.130514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:01:32.513699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130138+0000","last_change":"2024-04-26T19:50:31.103841+0000","last_active":"2024-04-26T19:50:37.130138+0000","last_peered":"2024-04-26T19:50:37.130138+0000","last_clean":"2024-04-26T19:50:37.130138+0000","last_became_active":"2024-04-26T19:50:31.103366+0000","last_became_peered":"2024-04-26T19:50:31.103366+0000","last_unstale":"2024-04-26T19:50:37.130138+0000","last_undegraded":"2024-04-26T19:50:37.130138+0000","last_fullsized":"2024-04-26T19:50:37.130138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:33:22.032597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653217+0000","last_change":"2024-04-26T19:50:31.113740+0000","last_active":"2024-04-26T19:50:38.653217+0000","last_peered":"2024-04-26T19:50:38.653217+0000","last_clean":"2024-04-26T19:50:38.653217+0000","last_became_active":"2024-04-26T19:50:31.113431+0000","last_became_peered":"2024-04-26T19:50:31.113431+0000","last_unstale":"2024-04-26T19:50:38.653217+0000","last_undegraded":"2024-04-26T19:50:38.653217+0000","last_fullsized":"2024-04-26T19:50:38.653217+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:57:15.210799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653594+0000","last_change":"2024-04-26T19:50:31.113198+0000","last_active":"2024-04-26T19:50:38.653594+0000","last_peered":"2024-04-26T19:50:38.653594+0000","last_clean":"2024-04-26T19:50:38.653594+0000","last_became_active":"2024-04-26T19:50:31.112582+0000","last_became_peered":"2024-04-26T19:50:31.112582+0000","last_unstale":"2024-04-26T19:50:38.653594+0000","last_undegraded":"2024-04-26T19:50:38.653594+0000","last_fullsized":"2024-04-26T19:50:38.653594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653995+0000","last_change":"2024-04-26T19:50:31.120475+0000","last_active":"2024-04-26T19:50:38.653995+0000","last_peered":"2024-04-26T19:50:38.653995+0000","last_clean":"2024-04-26T19:50:38.653995+0000","last_became_active":"2024-04-26T19:50:31.118576+0000","last_became_peered":"2024-04-26T19:50:31.118576+0000","last_unstale":"2024-04-26T19:50:38.653995+0000","last_undegraded":"2024-04-26T19:50:38.653995+0000","last_fullsized":"2024-04-26T19:50:38.653995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:30:06.291653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.130241+0000","last_change":"2024-04-26T19:50:31.111907+0000","last_active":"2024-04-26T19:50:37.130241+0000","last_peered":"2024-04-26T19:50:37.130241+0000","last_clean":"2024-04-26T19:50:37.130241+0000","last_became_active":"2024-04-26T19:50:31.111623+0000","last_became_peered":"2024-04-26T19:50:31.111623+0000","last_unstale":"2024-04-26T19:50:37.130241+0000","last_undegraded":"2024-04-26T19:50:37.130241+0000","last_fullsized":"2024-04-26T19:50:37.130241+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:06:38.754485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652974+0000","last_change":"2024-04-26T19:50:31.112347+0000","last_active":"2024-04-26T19:50:38.652974+0000","last_peered":"2024-04-26T19:50:38.652974+0000","last_clean":"2024-04-26T19:50:38.652974+0000","last_became_active":"2024-04-26T19:50:31.112074+0000","last_became_peered":"2024-04-26T19:50:31.112074+0000","last_unstale":"2024-04-26T19:50:38.652974+0000","last_undegraded":"2024-04-26T19:50:38.652974+0000","last_fullsized":"2024-04-26T19:50:38.652974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:54:56.756360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.654050+0000","last_change":"2024-04-26T19:50:31.115255+0000","last_active":"2024-04-26T19:50:38.654050+0000","last_peered":"2024-04-26T19:50:38.654050+0000","last_clean":"2024-04-26T19:50:38.654050+0000","last_became_active":"2024-04-26T19:50:31.114991+0000","last_became_peered":"2024-04-26T19:50:31.114991+0000","last_unstale":"2024-04-26T19:50:38.654050+0000","last_undegraded":"2024-04-26T19:50:38.654050+0000","last_fullsized":"2024-04-26T19:50:38.654050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:25:39.941264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.653558+0000","last_change":"2024-04-26T19:50:31.111763+0000","last_active":"2024-04-26T19:50:38.653558+0000","last_peered":"2024-04-26T19:50:38.653558+0000","last_clean":"2024-04-26T19:50:38.653558+0000","last_became_active":"2024-04-26T19:50:31.111487+0000","last_became_peered":"2024-04-26T19:50:31.111487+0000","last_unstale":"2024-04-26T19:50:38.653558+0000","last_undegraded":"2024-04-26T19:50:38.653558+0000","last_fullsized":"2024-04-26T19:50:38.653558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:13.577254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652454+0000","last_change":"2024-04-26T19:50:31.120637+0000","last_active":"2024-04-26T19:50:38.652454+0000","last_peered":"2024-04-26T19:50:38.652454+0000","last_clean":"2024-04-26T19:50:38.652454+0000","last_became_active":"2024-04-26T19:50:31.120388+0000","last_became_peered":"2024-04-26T19:50:31.120388+0000","last_unstale":"2024-04-26T19:50:38.652454+0000","last_undegraded":"2024-04-26T19:50:38.652454+0000","last_fullsized":"2024-04-26T19:50:38.652454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:43.955002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.299955+0000","last_change":"2024-04-26T19:50:31.104484+0000","last_active":"2024-04-26T19:50:38.299955+0000","last_peered":"2024-04-26T19:50:38.299955+0000","last_clean":"2024-04-26T19:50:38.299955+0000","last_became_active":"2024-04-26T19:50:31.104172+0000","last_became_peered":"2024-04-26T19:50:31.104172+0000","last_unstale":"2024-04-26T19:50:38.299955+0000","last_undegraded":"2024-04-26T19:50:38.299955+0000","last_fullsized":"2024-04-26T19:50:38.299955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:35:19.381417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:37.997884+0000","last_change":"2024-04-26T19:50:31.110923+0000","last_active":"2024-04-26T19:50:37.997884+0000","last_peered":"2024-04-26T19:50:37.997884+0000","last_clean":"2024-04-26T19:50:37.997884+0000","last_became_active":"2024-04-26T19:50:31.110601+0000","last_became_peered":"2024-04-26T19:50:31.110601+0000","last_unstale":"2024-04-26T19:50:37.997884+0000","last_undegraded":"2024-04-26T19:50:37.997884+0000","last_fullsized":"2024-04-26T19:50:37.997884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:12.810838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-26T19:50:38.652978+0000","last_change":"2024-04-26T19:50:31.112465+0000","last_active":"2024-04-26T19:50:38.652978+0000","last_peered":"2024-04-26T19:50:38.652978+0000","last_clean":"2024-04-26T19:50:38.652978+0000","last_became_active":"2024-04-26T19:50:31.112186+0000","last_became_peered":"2024-04-26T19:50:31.112186+0000","last_unstale":"2024-04-26T19:50:38.652978+0000","last_undegraded":"2024-04-26T19:50:38.652978+0000","last_fullsized":"2024-04-26T19:50:38.652978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:30.080131+0000","last_clean_scrub_stamp":"2024-04-26T19:50:30.080131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:10.399495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:50:58.857 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T19:50:58.857 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T19:50:58.857 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T19:50:58.857 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T19:50:58.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.987+0000 7fb8da139640 1 Processor -- start 2024-04-26T19:50:58.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.987+0000 7fb8da139640 1 -- start start 2024-04-26T19:50:58.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.987+0000 7fb8da139640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8d4072a30 0x7fb8d4072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.987+0000 7fb8da139640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d40b3de0 0x7fb8d40b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.987+0000 7fb8da139640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8d40734f0 con 0x7fb8d415d440 2024-04-26T19:50:58.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8da139640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb8d40737d0 con 0x7fb8d4072a30 2024-04-26T19:50:58.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8da139640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d40cde30 con 0x7fb8d40b3de0 2024-04-26T19:50:58.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8d3fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d40b3de0 0x7fb8d40b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8d2ffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb8d4072a30 0x7fb8d4072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8d37fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fb8d415d440 0x7fb8d415d820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:49256/0 (socket says 172.21.15.79:49256) 2024-04-26T19:50:58.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8d37fe640 1 -- 172.21.15.79:0/655842010 learned_addr learned my addr 172.21.15.79:0/655842010 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:58.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8d3fff640 1 -- 172.21.15.79:0/655842010 >> v2:172.21.15.83:3300/0 conn(0x7fb8d4072a30 msgr2=0x7fb8d4072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:58.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8d3fff640 1 --2- 172.21.15.79:0/655842010 >> v2:172.21.15.83:3300/0 conn(0x7fb8d4072a30 0x7fb8d4072e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8d3fff640 1 -- 172.21.15.79:0/655842010 >> v1:172.21.15.83:6789/0 conn(0x7fb8d415d440 legacy=0x7fb8d415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8d3fff640 1 -- 172.21.15.79:0/655842010 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d40b3230 con 0x7fb8d40b3de0 2024-04-26T19:50:58.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.988+0000 7fb8d2ffd640 1 --2- 172.21.15.79:0/655842010 >> v2:172.21.15.83:3300/0 conn(0x7fb8d4072a30 0x7fb8d4072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.989+0000 7fb8d3fff640 1 --2- 172.21.15.79:0/655842010 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d40b3de0 0x7fb8d40b4270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009d90 tx=0x7fb8cc02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7128d20c9972575 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.989+0000 7fb8d27fc640 1 -- 172.21.15.79:0/655842010 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc007740 con 0x7fb8d40b3de0 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.989+0000 7fb8d27fc640 1 -- 172.21.15.79:0/655842010 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8cc0078e0 con 0x7fb8d40b3de0 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.989+0000 7fb8d27fc640 1 -- 172.21.15.79:0/655842010 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8cc007cb0 con 0x7fb8d40b3de0 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.990+0000 7fb8da139640 1 -- 172.21.15.79:0/655842010 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d40b3de0 msgr2=0x7fb8d40b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.990+0000 7fb8da139640 1 --2- 172.21.15.79:0/655842010 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d40b3de0 0x7fb8d40b4270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009d90 tx=0x7fb8cc02a660 comp rx=0 tx=0).stop 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.990+0000 7fb8da139640 1 -- 172.21.15.79:0/655842010 shutdown_connections 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.990+0000 7fb8da139640 1 --2- 172.21.15.79:0/655842010 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d40b3de0 0x7fb8d40b4270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.990+0000 7fb8da139640 1 --2- 172.21.15.79:0/655842010 >> v2:172.21.15.83:3300/0 conn(0x7fb8d4072a30 0x7fb8d4072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.990+0000 7fb8da139640 1 -- 172.21.15.79:0/655842010 >> 172.21.15.79:0/655842010 conn(0x7fb8d40b39d0 msgr2=0x7fb8d406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.990+0000 7fb8da139640 1 -- 172.21.15.79:0/655842010 shutdown_connections 2024-04-26T19:50:58.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.990+0000 7fb8da139640 1 -- 172.21.15.79:0/655842010 wait complete. 2024-04-26T19:50:58.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.991+0000 7fb8da139640 1 Processor -- start 2024-04-26T19:50:58.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.991+0000 7fb8da139640 1 -- start start 2024-04-26T19:50:58.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8da139640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d4072a30 0x7fb8d40c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8da139640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d40b3de0 0x7fb8d414bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8da139640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d415d440 0x7fb8d414c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8da139640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d40ce7b0 con 0x7fb8d4072a30 2024-04-26T19:50:58.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8da139640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d40ce630 con 0x7fb8d40b3de0 2024-04-26T19:50:58.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8da139640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb8d40ce930 con 0x7fb8d415d440 2024-04-26T19:50:58.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d4072a30 0x7fb8d40c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d4072a30 0x7fb8d40c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46254/0 (socket says 172.21.15.79:46254) 2024-04-26T19:50:58.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d37fe640 1 -- 172.21.15.79:0/1495966517 learned_addr learned my addr 172.21.15.79:0/1495966517 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:58.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d3fff640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d415d440 0x7fb8d414c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d2ffd640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d40b3de0 0x7fb8d414bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d37fe640 1 -- 172.21.15.79:0/1495966517 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d415d440 msgr2=0x7fb8d414c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d37fe640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d415d440 0x7fb8d414c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d37fe640 1 -- 172.21.15.79:0/1495966517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d40b3de0 msgr2=0x7fb8d414bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d37fe640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d40b3de0 0x7fb8d414bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d37fe640 1 -- 172.21.15.79:0/1495966517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d40abfa0 con 0x7fb8d4072a30 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8d37fe640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d4072a30 0x7fb8d40c4ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c3c0 tx=0x7fb8c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.992+0000 7fb8bbfff640 1 -- 172.21.15.79:0/1495966517 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4017070 con 0x7fb8d4072a30 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.993+0000 7fb8bbfff640 1 -- 172.21.15.79:0/1495966517 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8c4005ce0 con 0x7fb8d4072a30 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.993+0000 7fb8bbfff640 1 -- 172.21.15.79:0/1495966517 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40072f0 con 0x7fb8d4072a30 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.993+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d40ab420 con 0x7fb8d4072a30 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.993+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d40ab9e0 con 0x7fb8d4072a30 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.994+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d40506c0 con 0x7fb8d4072a30 2024-04-26T19:50:58.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.994+0000 7fb8bbfff640 1 -- 172.21.15.79:0/1495966517 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb8c401a070 con 0x7fb8d4072a30 2024-04-26T19:50:58.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.994+0000 7fb8bbfff640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8c006c8f0 0x7fb8c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:58.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.995+0000 7fb8bbfff640 1 -- 172.21.15.79:0/1495966517 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb8c408b220 con 0x7fb8d4072a30 2024-04-26T19:50:58.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.995+0000 7fb8d2ffd640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8c006c8f0 0x7fb8c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:58.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.995+0000 7fb8d2ffd640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8c006c8f0 0x7fb8c006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009560 tx=0x7fb8c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:58.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:58.999+0000 7fb8bbfff640 1 -- 172.21.15.79:0/1495966517 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb8c4059320 con 0x7fb8d4072a30 2024-04-26T19:50:59.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.214+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb8d406e320 con 0x7fb8d4072a30 2024-04-26T19:50:59.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.214+0000 7fb8bbfff640 1 -- 172.21.15.79:0/1495966517 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb8c4058c20 con 0x7fb8d4072a30 2024-04-26T19:50:59.215 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:59.215 INFO:teuthology.orchestra.run.smithi079.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8c006c8f0 msgr2=0x7fb8c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8c006c8f0 0x7fb8c006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009560 tx=0x7fb8c8008040 comp rx=0 tx=0).stop 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d4072a30 msgr2=0x7fb8d40c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d4072a30 0x7fb8d40c4ee0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb8c400c3c0 tx=0x7fb8c400c880 comp rx=0 tx=0).stop 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 shutdown_connections 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb8c006c8f0 0x7fb8c006edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb8d415d440 0x7fb8d414c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb8d40b3de0 0x7fb8d414bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 --2- 172.21.15.79:0/1495966517 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb8d4072a30 0x7fb8d40c4ee0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 >> 172.21.15.79:0/1495966517 conn(0x7fb8d40b39d0 msgr2=0x7fb8d406fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:59.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 shutdown_connections 2024-04-26T19:50:59.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.216+0000 7fb8da139640 1 -- 172.21.15.79:0/1495966517 wait complete. 2024-04-26T19:50:59.228 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T19:50:59.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:50:59.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.324+0000 7f735f293640 1 Processor -- start 2024-04-26T19:50:59.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.324+0000 7f735f293640 1 -- start start 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735f293640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f73580b6c00 0x7f73580d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735f293640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73580d06a0 con 0x7f73580c79c0 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735f293640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73580d0980 con 0x7f7358145e20 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735f293640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f73580d0460 con 0x7f73580b6c00 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735d830640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f73580b6c00 0x7f73580d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735c82e640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f73580c79c0 0x7f73580b64f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53094/0 (socket says 172.21.15.79:53094) 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735c82e640 1 -- 172.21.15.79:0/561819148 learned_addr learned my addr 172.21.15.79:0/561819148 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735d830640 1 -- 172.21.15.79:0/561819148 >> v1:172.21.15.83:6789/0 conn(0x7f7358145e20 legacy=0x7f73580cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735d830640 1 -- 172.21.15.79:0/561819148 >> v1:172.21.15.79:6789/0 conn(0x7f73580c79c0 legacy=0x7f73580b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.325+0000 7f735d830640 1 -- 172.21.15.79:0/561819148 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73580d0c60 con 0x7f73580b6c00 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.326+0000 7f735d830640 1 --2- 172.21.15.79:0/561819148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580b6c00 0x7f73580d1740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73500097a0 tx=0x7f735002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32171840cc16fc78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.326+0000 7f734ffff640 1 -- 172.21.15.79:0/561819148 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350037070 con 0x7f73580b6c00 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.326+0000 7f734ffff640 1 -- 172.21.15.79:0/561819148 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f735002ae50 con 0x7f73580b6c00 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.326+0000 7f734ffff640 1 -- 172.21.15.79:0/561819148 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350032660 con 0x7f73580b6c00 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.327+0000 7f735f293640 1 -- 172.21.15.79:0/561819148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580b6c00 msgr2=0x7f73580d1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.327+0000 7f735f293640 1 --2- 172.21.15.79:0/561819148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580b6c00 0x7f73580d1740 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f73500097a0 tx=0x7f735002a140 comp rx=0 tx=0).stop 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.327+0000 7f735f293640 1 -- 172.21.15.79:0/561819148 shutdown_connections 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.327+0000 7f735f293640 1 --2- 172.21.15.79:0/561819148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580b6c00 0x7f73580d1740 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.327+0000 7f735f293640 1 -- 172.21.15.79:0/561819148 >> 172.21.15.79:0/561819148 conn(0x7f735806c770 msgr2=0x7f735806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.327+0000 7f735f293640 1 -- 172.21.15.79:0/561819148 shutdown_connections 2024-04-26T19:50:59.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.327+0000 7f735f293640 1 -- 172.21.15.79:0/561819148 wait complete. 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.328+0000 7f735f293640 1 Processor -- start 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.328+0000 7f735f293640 1 -- start start 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735f293640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f73580b6c00 0x7f7358141150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735f293640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580c79c0 0x7f73581e7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735f293640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7358145e20 0x7f73581e7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735f293640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f735815d660 con 0x7f73580c79c0 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735f293640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f73580d1220 con 0x7f7358145e20 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735f293640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f735815d7e0 con 0x7f73580b6c00 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735c82e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580c79c0 0x7f73581e7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735c82e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580c79c0 0x7f73581e7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46270/0 (socket says 172.21.15.79:46270) 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735c82e640 1 -- 172.21.15.79:0/108752448 learned_addr learned my addr 172.21.15.79:0/108752448 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735d830640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7358145e20 0x7f73581e7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735c82e640 1 -- 172.21.15.79:0/108752448 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f73580b6c00 msgr2=0x7f7358141150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735c82e640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f73580b6c00 0x7f7358141150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735c82e640 1 -- 172.21.15.79:0/108752448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7358145e20 msgr2=0x7f73581e7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735c82e640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7358145e20 0x7f73581e7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735c82e640 1 -- 172.21.15.79:0/108752448 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73580c9390 con 0x7f73580c79c0 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735d830640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7358145e20 0x7f73581e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735c82e640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580c79c0 0x7f73581e7100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f734800c400 tx=0x7f734800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f734dffb640 1 -- 172.21.15.79:0/108752448 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348017070 con 0x7f73580c79c0 2024-04-26T19:50:59.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f734dffb640 1 -- 172.21.15.79:0/108752448 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7348005ce0 con 0x7f73580c79c0 2024-04-26T19:50:59.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73580c9670 con 0x7f73580c79c0 2024-04-26T19:50:59.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.329+0000 7f734dffb640 1 -- 172.21.15.79:0/108752448 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73480072f0 con 0x7f73580c79c0 2024-04-26T19:50:59.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.330+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73580c9c30 con 0x7f73580c79c0 2024-04-26T19:50:59.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.331+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73580506c0 con 0x7f73580c79c0 2024-04-26T19:50:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.331+0000 7f734dffb640 1 -- 172.21.15.79:0/108752448 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f734801a070 con 0x7f73580c79c0 2024-04-26T19:50:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.331+0000 7f734dffb640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f733c06c8f0 0x7f733c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.331+0000 7f734dffb640 1 -- 172.21.15.79:0/108752448 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f734808b210 con 0x7f73580c79c0 2024-04-26T19:50:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.331+0000 7f735d02f640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f733c06c8f0 0x7f733c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.332+0000 7f735d02f640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f733c06c8f0 0x7f733c06edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f735400ab90 tx=0x7f7354009230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:59.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.335+0000 7f734dffb640 1 -- 172.21.15.79:0/108752448 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7348059390 con 0x7f73580c79c0 2024-04-26T19:50:59.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.509+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73580cfdf0 con 0x7f73580c79c0 2024-04-26T19:50:59.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.510+0000 7f734dffb640 1 -- 172.21.15.79:0/108752448 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7f7348058c90 con 0x7f73580c79c0 2024-04-26T19:50:59.511 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:50:59.511 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f733c06c8f0 msgr2=0x7f733c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f733c06c8f0 0x7f733c06edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f735400ab90 tx=0x7f7354009230 comp rx=0 tx=0).stop 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580c79c0 msgr2=0x7f73581e7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580c79c0 0x7f73581e7100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f734800c400 tx=0x7f734800c8c0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 shutdown_connections 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f733c06c8f0 0x7f733c06edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7358145e20 0x7f73581e7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f73580c79c0 0x7f73581e7100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 --2- 172.21.15.79:0/108752448 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f73580b6c00 0x7f7358141150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 >> 172.21.15.79:0/108752448 conn(0x7f735806c770 msgr2=0x7f73580b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:59.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 shutdown_connections 2024-04-26T19:50:59.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.512+0000 7f735f293640 1 -- 172.21.15.79:0/108752448 wait complete. 2024-04-26T19:50:59.524 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:50:59.524 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:50:59.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.624+0000 7fa9d67e4640 1 Processor -- start 2024-04-26T19:50:59.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.625+0000 7fa9d67e4640 1 -- start start 2024-04-26T19:50:59.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9d67e4640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9d006c350 0x7fa9d006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9d67e4640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d00b7e60 con 0x7fa9d00b6630 2024-04-26T19:50:59.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9d67e4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9d00b8140 con 0x7fa9d0148f30 2024-04-26T19:50:59.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9d67e4640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fa9d00b6a70 con 0x7fa9d006c350 2024-04-26T19:50:59.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9d4d81640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9d006c350 0x7fa9d006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9cf7fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa9d00b6630 0x7fa9d00b8f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53110/0 (socket says 172.21.15.79:53110) 2024-04-26T19:50:59.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9cffff640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa9d0148f30 0x7fa9d00b7620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:49270/0 (socket says 172.21.15.79:49270) 2024-04-26T19:50:59.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9cf7fe640 1 -- 172.21.15.79:0/3885762110 learned_addr learned my addr 172.21.15.79:0/3885762110 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:50:59.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9d4d81640 1 -- 172.21.15.79:0/3885762110 >> v1:172.21.15.83:6789/0 conn(0x7fa9d0148f30 legacy=0x7fa9d00b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9d4d81640 1 -- 172.21.15.79:0/3885762110 >> v1:172.21.15.79:6789/0 conn(0x7fa9d00b6630 legacy=0x7fa9d00b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9d4d81640 1 -- 172.21.15.79:0/3885762110 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d00b8420 con 0x7fa9d006c350 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.626+0000 7fa9ceffd640 1 -- 172.21.15.79:0/3885762110 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4241761859 0 0) 0x7fa9d00b7e60 con 0x7fa9d00b6630 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.627+0000 7fa9d4d81640 1 --2- 172.21.15.79:0/3885762110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 0x7fa9d006c7e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009d90 tx=0x7fa9c802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b2609dcd0288a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.627+0000 7fa9ceffd640 1 -- 172.21.15.79:0/3885762110 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8007740 con 0x7fa9d006c350 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.627+0000 7fa9ceffd640 1 -- 172.21.15.79:0/3885762110 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c80078e0 con 0x7fa9d006c350 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.627+0000 7fa9ceffd640 1 -- 172.21.15.79:0/3885762110 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8007cb0 con 0x7fa9d006c350 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.628+0000 7fa9d67e4640 1 -- 172.21.15.79:0/3885762110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 msgr2=0x7fa9d006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.628+0000 7fa9d67e4640 1 --2- 172.21.15.79:0/3885762110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 0x7fa9d006c7e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009d90 tx=0x7fa9c802a660 comp rx=0 tx=0).stop 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.628+0000 7fa9d67e4640 1 -- 172.21.15.79:0/3885762110 shutdown_connections 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.628+0000 7fa9d67e4640 1 --2- 172.21.15.79:0/3885762110 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 0x7fa9d006c7e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.628+0000 7fa9d67e4640 1 -- 172.21.15.79:0/3885762110 >> 172.21.15.79:0/3885762110 conn(0x7fa9d0073b40 msgr2=0x7fa9d0073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.628+0000 7fa9d67e4640 1 -- 172.21.15.79:0/3885762110 shutdown_connections 2024-04-26T19:50:59.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.628+0000 7fa9d67e4640 1 -- 172.21.15.79:0/3885762110 wait complete. 2024-04-26T19:50:59.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.629+0000 7fa9d67e4640 1 Processor -- start 2024-04-26T19:50:59.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.629+0000 7fa9d67e4640 1 -- start start 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9d67e4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 0x7fa9d013ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9d67e4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9d00b6630 0x7fa9d0141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9d67e4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d0148f30 0x7fa9d01f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9d67e4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d015fbf0 con 0x7fa9d006c350 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9d67e4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d00b89e0 con 0x7fa9d0148f30 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9d67e4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d015fd70 con 0x7fa9d00b6630 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 0x7fa9d013ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 0x7fa9d013ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46282/0 (socket says 172.21.15.79:46282) 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cffff640 1 -- 172.21.15.79:0/69875433 learned_addr learned my addr 172.21.15.79:0/69875433 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9d4d81640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d0148f30 0x7fa9d01f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cf7fe640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9d00b6630 0x7fa9d0141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cffff640 1 -- 172.21.15.79:0/69875433 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9d00b6630 msgr2=0x7fa9d0141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cffff640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9d00b6630 0x7fa9d0141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cffff640 1 -- 172.21.15.79:0/69875433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d0148f30 msgr2=0x7fa9d01f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cffff640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d0148f30 0x7fa9d01f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cffff640 1 -- 172.21.15.79:0/69875433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d013b390 con 0x7fa9d006c350 2024-04-26T19:50:59.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.630+0000 7fa9cffff640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 0x7fa9d013ff70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c3e0 tx=0x7fa9c000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:59.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.631+0000 7fa9ccff9640 1 -- 172.21.15.79:0/69875433 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00074e0 con 0x7fa9d006c350 2024-04-26T19:50:59.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.631+0000 7fa9ccff9640 1 -- 172.21.15.79:0/69875433 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9c0007680 con 0x7fa9d006c350 2024-04-26T19:50:59.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.631+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d013a2f0 con 0x7fa9d006c350 2024-04-26T19:50:59.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.631+0000 7fa9ccff9640 1 -- 172.21.15.79:0/69875433 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9c00049e0 con 0x7fa9d006c350 2024-04-26T19:50:59.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.631+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d013a8b0 con 0x7fa9d006c350 2024-04-26T19:50:59.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.632+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d00506c0 con 0x7fa9d006c350 2024-04-26T19:50:59.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.632+0000 7fa9ccff9640 1 -- 172.21.15.79:0/69875433 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa9c001a020 con 0x7fa9d006c350 2024-04-26T19:50:59.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.633+0000 7fa9ccff9640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa9bc06c920 0x7fa9bc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.633+0000 7fa9ccff9640 1 -- 172.21.15.79:0/69875433 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa9c008b480 con 0x7fa9d006c350 2024-04-26T19:50:59.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.633+0000 7fa9cf7fe640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa9bc06c920 0x7fa9bc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.633+0000 7fa9cf7fe640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa9bc06c920 0x7fa9bc06ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4007f70 tx=0x7fa9c4007f00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:59.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.637+0000 7fa9ccff9640 1 -- 172.21.15.79:0/69875433 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa9c0058fb0 con 0x7fa9d006c350 2024-04-26T19:50:59.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.811+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa9d00b7440 con 0x7fa9d006c350 2024-04-26T19:50:59.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.811+0000 7fa9ccff9640 1 -- 172.21.15.79:0/69875433 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fa9c005d2b0 con 0x7fa9d006c350 2024-04-26T19:50:59.811 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa9bc06c920 msgr2=0x7fa9bc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa9bc06c920 0x7fa9bc06ede0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4007f70 tx=0x7fa9c4007f00 comp rx=0 tx=0).stop 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 msgr2=0x7fa9d013ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 0x7fa9d013ff70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa9c000c3e0 tx=0x7fa9c000c8a0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 shutdown_connections 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa9bc06c920 0x7fa9bc06ede0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9d0148f30 0x7fa9d01f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9d00b6630 0x7fa9d0141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 --2- 172.21.15.79:0/69875433 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9d006c350 0x7fa9d013ff70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 >> 172.21.15.79:0/69875433 conn(0x7fa9d0073b40 msgr2=0x7fa9d013cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:59.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 shutdown_connections 2024-04-26T19:50:59.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.814+0000 7fa9d67e4640 1 -- 172.21.15.79:0/69875433 wait complete. 2024-04-26T19:50:59.824 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:50:59.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.924+0000 7fe52e189640 1 Processor -- start 2024-04-26T19:50:59.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.924+0000 7fe52e189640 1 -- start start 2024-04-26T19:50:59.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe52e189640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe5280cf820 0x7fe5280741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe52e189640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe52806f7f0 0x7fe5280d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe52e189640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe528142b90 con 0x7fe528074720 2024-04-26T19:50:59.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe52e189640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe5281406e0 con 0x7fe52806f7f0 2024-04-26T19:50:59.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe52e189640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe528142e70 con 0x7fe5280cf820 2024-04-26T19:50:59.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe5277fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe5280cf820 0x7fe5280741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe527fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe52806f7f0 0x7fe5280d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe5277fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe5280cf820 0x7fe5280741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:46724/0 (socket says 172.21.15.79:46724) 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe527fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe52806f7f0 0x7fe5280d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:36362/0 (socket says 172.21.15.79:36362) 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe5277fe640 1 -- 172.21.15.79:0/588277998 learned_addr learned my addr 172.21.15.79:0/588277998 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe526ffd640 1 --1- 172.21.15.79:0/588277998 >> v1:172.21.15.83:6789/0 conn(0x7fe528074720 0x7fe52806f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:49276/0 (socket says 172.21.15.79:49276) 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe527fff640 1 -- 172.21.15.79:0/588277998 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe5280cf820 msgr2=0x7fe5280741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe527fff640 1 --2- 172.21.15.79:0/588277998 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe5280cf820 0x7fe5280741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe527fff640 1 -- 172.21.15.79:0/588277998 >> v1:172.21.15.83:6789/0 conn(0x7fe528074720 legacy=0x7fe52806f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe527fff640 1 -- 172.21.15.79:0/588277998 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe528143150 con 0x7fe52806f7f0 2024-04-26T19:50:59.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.925+0000 7fe5277fe640 1 --2- 172.21.15.79:0/588277998 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe5280cf820 0x7fe5280741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:50:59.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.926+0000 7fe527fff640 1 --2- 172.21.15.79:0/588277998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe52806f7f0 0x7fe5280d0980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe520009d90 tx=0x7fe52002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea9f7a616dbd077c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:59.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.926+0000 7fe5267fc640 1 -- 172.21.15.79:0/588277998 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520007740 con 0x7fe52806f7f0 2024-04-26T19:50:59.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.926+0000 7fe5267fc640 1 -- 172.21.15.79:0/588277998 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5200078e0 con 0x7fe52806f7f0 2024-04-26T19:50:59.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.926+0000 7fe5267fc640 1 -- 172.21.15.79:0/588277998 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520007cb0 con 0x7fe52806f7f0 2024-04-26T19:50:59.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.927+0000 7fe52e189640 1 -- 172.21.15.79:0/588277998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe52806f7f0 msgr2=0x7fe5280d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.927+0000 7fe52e189640 1 --2- 172.21.15.79:0/588277998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe52806f7f0 0x7fe5280d0980 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe520009d90 tx=0x7fe52002a660 comp rx=0 tx=0).stop 2024-04-26T19:50:59.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.927+0000 7fe52e189640 1 -- 172.21.15.79:0/588277998 shutdown_connections 2024-04-26T19:50:59.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.927+0000 7fe52e189640 1 --2- 172.21.15.79:0/588277998 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe52806f7f0 0x7fe5280d0980 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.927+0000 7fe52e189640 1 --2- 172.21.15.79:0/588277998 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe5280cf820 0x7fe5280741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.927+0000 7fe52e189640 1 -- 172.21.15.79:0/588277998 >> 172.21.15.79:0/588277998 conn(0x7fe5280689e0 msgr2=0x7fe5280b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:50:59.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.927+0000 7fe52e189640 1 -- 172.21.15.79:0/588277998 shutdown_connections 2024-04-26T19:50:59.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.927+0000 7fe52e189640 1 -- 172.21.15.79:0/588277998 wait complete. 2024-04-26T19:50:59.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.928+0000 7fe52e189640 1 Processor -- start 2024-04-26T19:50:59.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.928+0000 7fe52e189640 1 -- start start 2024-04-26T19:50:59.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe52e189640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe52806f7f0 0x7fe5280b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe52e189640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe528074720 0x7fe528159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe52e189640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5280cf820 0x7fe528160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe52e189640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe5280d1ee0 con 0x7fe52806f7f0 2024-04-26T19:50:59.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe52e189640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe528143710 con 0x7fe5280cf820 2024-04-26T19:50:59.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe52e189640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe5280d2060 con 0x7fe528074720 2024-04-26T19:50:59.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe5277fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe52806f7f0 0x7fe5280b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe5277fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe52806f7f0 0x7fe5280b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46288/0 (socket says 172.21.15.79:46288) 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe5277fe640 1 -- 172.21.15.79:0/4096068594 learned_addr learned my addr 172.21.15.79:0/4096068594 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe527fff640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5280cf820 0x7fe528160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe526ffd640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe528074720 0x7fe528159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe5277fe640 1 -- 172.21.15.79:0/4096068594 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe528074720 msgr2=0x7fe528159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe5277fe640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe528074720 0x7fe528159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe5277fe640 1 -- 172.21.15.79:0/4096068594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5280cf820 msgr2=0x7fe528160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe5277fe640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5280cf820 0x7fe528160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe5277fe640 1 -- 172.21.15.79:0/4096068594 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe528160880 con 0x7fe52806f7f0 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.929+0000 7fe527fff640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5280cf820 0x7fe528160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.930+0000 7fe5277fe640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe52806f7f0 0x7fe5280b1ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe51800e9e0 tx=0x7fe51800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.930+0000 7fe50ffff640 1 -- 172.21.15.79:0/4096068594 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518017070 con 0x7fe52806f7f0 2024-04-26T19:50:59.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.930+0000 7fe50ffff640 1 -- 172.21.15.79:0/4096068594 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51800c9d0 con 0x7fe52806f7f0 2024-04-26T19:50:59.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.930+0000 7fe50ffff640 1 -- 172.21.15.79:0/4096068594 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51800cd80 con 0x7fe52806f7f0 2024-04-26T19:50:59.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.930+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5280b2550 con 0x7fe52806f7f0 2024-04-26T19:50:59.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.930+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52815c3f0 con 0x7fe52806f7f0 2024-04-26T19:50:59.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.931+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5280506c0 con 0x7fe52806f7f0 2024-04-26T19:50:59.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.931+0000 7fe50ffff640 1 -- 172.21.15.79:0/4096068594 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe51801a070 con 0x7fe52806f7f0 2024-04-26T19:50:59.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.932+0000 7fe50ffff640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe51406c8f0 0x7fe51406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:50:59.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.932+0000 7fe526ffd640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe51406c8f0 0x7fe51406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:50:59.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.932+0000 7fe50ffff640 1 -- 172.21.15.79:0/4096068594 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe51808bc70 con 0x7fe52806f7f0 2024-04-26T19:50:59.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.932+0000 7fe526ffd640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe51406c8f0 0x7fe51406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b3a80 tx=0x7fe51c007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:50:59.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:50:59.936+0000 7fe50ffff640 1 -- 172.21.15.79:0/4096068594 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe518059df0 con 0x7fe52806f7f0 2024-04-26T19:51:00.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.111+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe5280b4590 con 0x7fe52806f7f0 2024-04-26T19:51:00.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.111+0000 7fe50ffff640 1 -- 172.21.15.79:0/4096068594 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fe5180596f0 con 0x7fe52806f7f0 2024-04-26T19:51:00.111 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:00.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe51406c8f0 msgr2=0x7fe51406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe51406c8f0 0x7fe51406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe5280b3a80 tx=0x7fe51c007cc0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe52806f7f0 msgr2=0x7fe5280b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe52806f7f0 0x7fe5280b1ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe51800e9e0 tx=0x7fe51800eea0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 shutdown_connections 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe51406c8f0 0x7fe51406edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe5280cf820 0x7fe528160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe528074720 0x7fe528159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 --2- 172.21.15.79:0/4096068594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe52806f7f0 0x7fe5280b1ee0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 >> 172.21.15.79:0/4096068594 conn(0x7fe5280689e0 msgr2=0x7fe528160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 shutdown_connections 2024-04-26T19:51:00.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.114+0000 7fe52e189640 1 -- 172.21.15.79:0/4096068594 wait complete. 2024-04-26T19:51:00.124 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:00.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.225+0000 7f61f355f640 1 Processor -- start 2024-04-26T19:51:00.225 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.225+0000 7f61f355f640 1 -- start start 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61f355f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f61ec0b3480 0x7f61ec0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61f355f640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61ec0b7c60 con 0x7f61ec146bc0 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61f355f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61ec0b93a0 con 0x7f61ec0d2d40 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61f355f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f61ec0b3e10 con 0x7f61ec0b3480 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61f12fb640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f61ec146bc0 0x7f61ec0b6e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53114/0 (socket says 172.21.15.79:53114) 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61f12fb640 1 -- 172.21.15.79:0/1334421756 learned_addr learned my addr 172.21.15.79:0/1334421756 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61f1afc640 1 --2- 172.21.15.79:0/1334421756 >> v2:172.21.15.83:3300/0 conn(0x7f61ec0b3480 0x7f61ec0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 25171873 0 0) 0x7f61ec0b7c60 con 0x7f61ec146bc0 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61d8003520 con 0x7f61ec146bc0 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3243226557 0 0) 0x7f61d8003520 con 0x7f61ec146bc0 2024-04-26T19:51:00.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 >> v2:172.21.15.83:3300/0 conn(0x7f61ec0b3480 msgr2=0x7f61ec0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 --2- 172.21.15.79:0/1334421756 >> v2:172.21.15.83:3300/0 conn(0x7f61ec0b3480 0x7f61ec0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 >> v1:172.21.15.83:6789/0 conn(0x7f61ec0d2d40 legacy=0x7f61ec0b8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec0b7ed0 con 0x7f61ec146bc0 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f61e8001670 con 0x7f61ec146bc0 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 >> v1:172.21.15.79:6789/0 conn(0x7f61ec146bc0 legacy=0x7f61ec0b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61d80039b0 0x7f61d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d80043b0 0x7f61d8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61d8005080 0x7f61d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec0b3e10 con 0x7f61d8005080 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61d800bfe0 con 0x7f61d80043b0 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f61d800c1e0 con 0x7f61d80039b0 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.226+0000 7f61f1afc640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61d8005080 0x7f61d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.227+0000 7f61f0afa640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d80043b0 0x7f61d8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.227+0000 7f61f12fb640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61d80039b0 0x7f61d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.227+0000 7f61f1afc640 1 -- 172.21.15.79:0/1334421756 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61d80039b0 msgr2=0x7f61d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.227+0000 7f61f1afc640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61d80039b0 0x7f61d8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.227+0000 7f61f1afc640 1 -- 172.21.15.79:0/1334421756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d80043b0 msgr2=0x7f61d8004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.227+0000 7f61f1afc640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d80043b0 0x7f61d8004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.227+0000 7f61f1afc640 1 -- 172.21.15.79:0/1334421756 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d8003520 con 0x7f61d8005080 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.227+0000 7f61f1afc640 1 -- 172.21.15.79:0/1334421756 reap_dead start 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.227+0000 7f61f1afc640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61d8005080 0x7f61d800b9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009fc0 tx=0x7f61d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181578de730f262c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4035020 con 0x7f61d8005080 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61e4003bd0 con 0x7f61d8005080 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61e3fff640 1 -- 172.21.15.79:0/1334421756 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e4009c10 con 0x7f61d8005080 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61f355f640 1 -- 172.21.15.79:0/1334421756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61d8005080 msgr2=0x7f61d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61f355f640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61d8005080 0x7f61d800b9c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f61e4009fc0 tx=0x7f61d8005830 comp rx=0 tx=0).stop 2024-04-26T19:51:00.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61f355f640 1 -- 172.21.15.79:0/1334421756 shutdown_connections 2024-04-26T19:51:00.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61f355f640 1 --2- 172.21.15.79:0/1334421756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61d8005080 0x7f61d800b9c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61f355f640 1 -- 172.21.15.79:0/1334421756 >> 172.21.15.79:0/1334421756 conn(0x7f61ec0b4500 msgr2=0x7f61ec0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:00.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61f355f640 1 -- 172.21.15.79:0/1334421756 shutdown_connections 2024-04-26T19:51:00.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.228+0000 7f61f355f640 1 -- 172.21.15.79:0/1334421756 wait complete. 2024-04-26T19:51:00.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.229+0000 7f61f355f640 1 Processor -- start 2024-04-26T19:51:00.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.229+0000 7f61f355f640 1 -- start start 2024-04-26T19:51:00.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f355f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d8005080 0x7f61ec142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f355f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61ec15ac70 0x7f61ec15b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f355f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61ec1f7f20 0x7f61ec1f83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f355f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec0b8770 con 0x7f61ec15ac70 2024-04-26T19:51:00.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f355f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f61ec0b8490 con 0x7f61d8005080 2024-04-26T19:51:00.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f355f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f61ec146fa0 con 0x7f61ec1f7f20 2024-04-26T19:51:00.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f0afa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61ec15ac70 0x7f61ec15b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f0afa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61ec15ac70 0x7f61ec15b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46312/0 (socket says 172.21.15.79:46312) 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f0afa640 1 -- 172.21.15.79:0/2284005252 learned_addr learned my addr 172.21.15.79:0/2284005252 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f12fb640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d8005080 0x7f61ec142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f1afc640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61ec1f7f20 0x7f61ec1f83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f0afa640 1 -- 172.21.15.79:0/2284005252 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61ec1f7f20 msgr2=0x7f61ec1f83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f0afa640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61ec1f7f20 0x7f61ec1f83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f0afa640 1 -- 172.21.15.79:0/2284005252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d8005080 msgr2=0x7f61ec142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f0afa640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d8005080 0x7f61ec142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f0afa640 1 -- 172.21.15.79:0/2284005252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ec1f8c50 con 0x7f61ec15ac70 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.230+0000 7f61f12fb640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d8005080 0x7f61ec142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.231+0000 7f61f0afa640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61ec15ac70 0x7f61ec15b0b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00f7c0 tx=0x7f61dc00fc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.231+0000 7f61e1ffb640 1 -- 172.21.15.79:0/2284005252 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00fe70 con 0x7f61ec15ac70 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.231+0000 7f61e1ffb640 1 -- 172.21.15.79:0/2284005252 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61dc018070 con 0x7f61ec15ac70 2024-04-26T19:51:00.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.231+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ec15b5f0 con 0x7f61ec15ac70 2024-04-26T19:51:00.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.231+0000 7f61e1ffb640 1 -- 172.21.15.79:0/2284005252 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00d730 con 0x7f61ec15ac70 2024-04-26T19:51:00.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.231+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ec0b0530 con 0x7f61ec15ac70 2024-04-26T19:51:00.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.232+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ec0506c0 con 0x7f61ec15ac70 2024-04-26T19:51:00.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.232+0000 7f61e1ffb640 1 -- 172.21.15.79:0/2284005252 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f61dc00d8d0 con 0x7f61ec15ac70 2024-04-26T19:51:00.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.232+0000 7f61e1ffb640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f61d006c8f0 0x7f61d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.233+0000 7f61e1ffb640 1 -- 172.21.15.79:0/2284005252 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f61dc08bda0 con 0x7f61ec15ac70 2024-04-26T19:51:00.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.233+0000 7f61f12fb640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f61d006c8f0 0x7f61d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.233+0000 7f61f12fb640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f61d006c8f0 0x7f61d006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f61e8007cb0 tx=0x7f61e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:00.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.237+0000 7f61e1ffb640 1 -- 172.21.15.79:0/2284005252 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f61dc01a310 con 0x7f61ec15ac70 2024-04-26T19:51:00.376 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:51:00.379 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:51:00.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.413+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f61ec076060 con 0x7f61ec15ac70 2024-04-26T19:51:00.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.414+0000 7f61e1ffb640 1 -- 172.21.15.79:0/2284005252 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f61dc059e90 con 0x7f61ec15ac70 2024-04-26T19:51:00.413 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f61d006c8f0 msgr2=0x7f61d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f61d006c8f0 0x7f61d006edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f61e8007cb0 tx=0x7f61e8009040 comp rx=0 tx=0).stop 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61ec15ac70 msgr2=0x7f61ec15b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61ec15ac70 0x7f61ec15b0b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00f7c0 tx=0x7f61dc00fc80 comp rx=0 tx=0).stop 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 shutdown_connections 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f61d006c8f0 0x7f61d006edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f61ec1f7f20 0x7f61ec1f83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f61ec15ac70 0x7f61ec15b0b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 --2- 172.21.15.79:0/2284005252 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f61d8005080 0x7f61ec142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 >> 172.21.15.79:0/2284005252 conn(0x7f61ec0b4500 msgr2=0x7f61ec0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 shutdown_connections 2024-04-26T19:51:00.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.416+0000 7f61f355f640 1 -- 172.21.15.79:0/2284005252 wait complete. 2024-04-26T19:51:00.426 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:00.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.567+0000 7f451a847640 1 Processor -- start 2024-04-26T19:51:00.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.567+0000 7f451a847640 1 -- start start 2024-04-26T19:51:00.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f451a847640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f451415d8c0 0x7f45140b6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f451a847640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f45140d1380 0x7f4514161430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f451a847640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45140b8c70 con 0x7f45140cfea0 2024-04-26T19:51:00.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f451a847640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4514157e50 con 0x7f45140d1380 2024-04-26T19:51:00.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f451a847640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4514160fa0 con 0x7f451415d8c0 2024-04-26T19:51:00.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f4513fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f45140cfea0 0x7f45140d0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53122/0 (socket says 172.21.15.79:53122) 2024-04-26T19:51:00.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f4513fff640 1 -- 172.21.15.79:0/3437045064 learned_addr learned my addr 172.21.15.79:0/3437045064 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:00.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f4518de4640 1 --2- 172.21.15.79:0/3437045064 >> v2:172.21.15.83:3300/0 conn(0x7f45140d1380 0x7f4514161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f45137fe640 1 --2- 172.21.15.79:0/3437045064 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f451415d8c0 0x7f45140b6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f4512ffd640 1 -- 172.21.15.79:0/3437045064 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 110135113 0 0) 0x7f4504001110 con 0x7f45140cfea0 2024-04-26T19:51:00.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f4512ffd640 1 -- 172.21.15.79:0/3437045064 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44fc003520 con 0x7f45140cfea0 2024-04-26T19:51:00.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f4518de4640 1 -- 172.21.15.79:0/3437045064 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f451415d8c0 msgr2=0x7f45140b6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:00.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f4518de4640 1 --2- 172.21.15.79:0/3437045064 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f451415d8c0 0x7f45140b6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f4518de4640 1 -- 172.21.15.79:0/3437045064 >> v1:172.21.15.79:6789/0 conn(0x7f45140cfea0 legacy=0x7f45140d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.568+0000 7f4518de4640 1 -- 172.21.15.79:0/3437045064 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451415df30 con 0x7f45140d1380 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.569+0000 7f4518de4640 1 --2- 172.21.15.79:0/3437045064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140d1380 0x7f4514161430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f450c009d90 tx=0x7f450c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e10c05f0348fb6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.569+0000 7f4512ffd640 1 -- 172.21.15.79:0/3437045064 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c007740 con 0x7f45140d1380 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.569+0000 7f4512ffd640 1 -- 172.21.15.79:0/3437045064 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c0078e0 con 0x7f45140d1380 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.569+0000 7f4512ffd640 1 -- 172.21.15.79:0/3437045064 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c007c90 con 0x7f45140d1380 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.570+0000 7f451a847640 1 -- 172.21.15.79:0/3437045064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140d1380 msgr2=0x7f4514161430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.570+0000 7f451a847640 1 --2- 172.21.15.79:0/3437045064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140d1380 0x7f4514161430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f450c009d90 tx=0x7f450c02a660 comp rx=0 tx=0).stop 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.570+0000 7f451a847640 1 -- 172.21.15.79:0/3437045064 shutdown_connections 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.570+0000 7f451a847640 1 --2- 172.21.15.79:0/3437045064 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140d1380 0x7f4514161430 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.570+0000 7f451a847640 1 --2- 172.21.15.79:0/3437045064 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f451415d8c0 0x7f45140b6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.570+0000 7f451a847640 1 -- 172.21.15.79:0/3437045064 >> 172.21.15.79:0/3437045064 conn(0x7f4514073b40 msgr2=0x7f4514073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.570+0000 7f451a847640 1 -- 172.21.15.79:0/3437045064 shutdown_connections 2024-04-26T19:51:00.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.570+0000 7f451a847640 1 -- 172.21.15.79:0/3437045064 wait complete. 2024-04-26T19:51:00.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.571+0000 7f451a847640 1 Processor -- start 2024-04-26T19:51:00.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.571+0000 7f451a847640 1 -- start start 2024-04-26T19:51:00.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f451a847640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140cfea0 0x7f4514156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f451a847640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45140d1380 0x7f45141413a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f451a847640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f451415d8c0 0x7f4514159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f451a847640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f451415e7d0 con 0x7f451415d8c0 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f451a847640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f451415e4f0 con 0x7f45140cfea0 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f451a847640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f45141406c0 con 0x7f45140d1380 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4518de4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f451415d8c0 0x7f4514159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4518de4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f451415d8c0 0x7f4514159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46324/0 (socket says 172.21.15.79:46324) 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4518de4640 1 -- 172.21.15.79:0/4035211307 learned_addr learned my addr 172.21.15.79:0/4035211307 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f45137fe640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45140d1380 0x7f45141413a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4513fff640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140cfea0 0x7f4514156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4518de4640 1 -- 172.21.15.79:0/4035211307 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45140d1380 msgr2=0x7f45141413a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4518de4640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45140d1380 0x7f45141413a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4518de4640 1 -- 172.21.15.79:0/4035211307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140cfea0 msgr2=0x7f4514156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4518de4640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140cfea0 0x7f4514156ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4518de4640 1 -- 172.21.15.79:0/4035211307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4514141d40 con 0x7f451415d8c0 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.572+0000 7f4513fff640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140cfea0 0x7f4514156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:00.576 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.573+0000 7f4518de4640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f451415d8c0 0x7f4514159bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f450c02ac50 tx=0x7f450c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.573+0000 7f4510ff9640 1 -- 172.21.15.79:0/4035211307 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c040070 con 0x7f451415d8c0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.573+0000 7f4510ff9640 1 -- 172.21.15.79:0/4035211307 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f450c031c60 con 0x7f451415d8c0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.573+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4514142020 con 0x7f451415d8c0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.573+0000 7f4510ff9640 1 -- 172.21.15.79:0/4035211307 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c009450 con 0x7f451415d8c0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.573+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45141425e0 con 0x7f451415d8c0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.574+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4514143ef0 con 0x7f451415d8c0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.574+0000 7f4510ff9640 1 -- 172.21.15.79:0/4035211307 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f450c039040 con 0x7f451415d8c0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.575+0000 7f4510ff9640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f450006c8f0 0x7f450006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.575+0000 7f4510ff9640 1 -- 172.21.15.79:0/4035211307 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f450c0ad200 con 0x7f451415d8c0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.575+0000 7f4513fff640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f450006c8f0 0x7f450006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.575+0000 7f4513fff640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f450006c8f0 0x7f450006edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4504007fd0 tx=0x7f4504009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:00.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.579+0000 7f4510ff9640 1 -- 172.21.15.79:0/4035211307 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f450c07b2f0 con 0x7f451415d8c0 2024-04-26T19:51:00.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.752+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f451415d460 con 0x7f451415d8c0 2024-04-26T19:51:00.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.752+0000 7f4510ff9640 1 -- 172.21.15.79:0/4035211307 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f450c07abf0 con 0x7f451415d8c0 2024-04-26T19:51:00.753 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f450006c8f0 msgr2=0x7f450006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f450006c8f0 0x7f450006edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4504007fd0 tx=0x7f4504009040 comp rx=0 tx=0).stop 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f451415d8c0 msgr2=0x7f4514159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f451415d8c0 0x7f4514159bb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f450c02ac50 tx=0x7f450c004a40 comp rx=0 tx=0).stop 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 shutdown_connections 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f450006c8f0 0x7f450006edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f451415d8c0 0x7f4514159bb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f45140d1380 0x7f45141413a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 --2- 172.21.15.79:0/4035211307 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f45140cfea0 0x7f4514156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 >> 172.21.15.79:0/4035211307 conn(0x7f4514073b40 msgr2=0x7f45140b7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 shutdown_connections 2024-04-26T19:51:00.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.754+0000 7f451a847640 1 -- 172.21.15.79:0/4035211307 wait complete. 2024-04-26T19:51:00.766 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:51:00.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.862+0000 7fd7fb5e6640 1 Processor -- start 2024-04-26T19:51:00.863 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.862+0000 7fd7fb5e6640 1 -- start start 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7fb5e6640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd7f4160390 0x7fd7f40b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7fb5e6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd7f4160770 0x7fd7f40b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7fb5e6640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f41630d0 0x7fd7f4163560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7fb5e6640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f4144ca0 con 0x7fd7f4160390 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7fb5e6640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd7f4163e00 con 0x7fd7f4160770 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7fb5e6640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f4144f30 con 0x7fd7f41630d0 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7f9382640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd7f4160390 0x7fd7f40b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7f9382640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd7f4160390 0x7fd7f40b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46334/0 (socket says 172.21.15.79:46334) 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7f9382640 1 -- 172.21.15.79:0/289788726 learned_addr learned my addr 172.21.15.79:0/289788726 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7f8b81640 1 --2- 172.21.15.79:0/289788726 >> v2:172.21.15.83:3300/0 conn(0x7fd7f4160770 0x7fd7f40b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.865 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7f9b83640 1 --2- 172.21.15.79:0/289788726 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f41630d0 0x7fd7f4163560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7f9382640 1 -- 172.21.15.79:0/289788726 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f41630d0 msgr2=0x7fd7f4163560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.863+0000 7fd7f9382640 1 --2- 172.21.15.79:0/289788726 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f41630d0 0x7fd7f4163560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.864+0000 7fd7f9382640 1 -- 172.21.15.79:0/289788726 >> v2:172.21.15.83:3300/0 conn(0x7fd7f4160770 msgr2=0x7fd7f40b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.864+0000 7fd7f9382640 1 --2- 172.21.15.79:0/289788726 >> v2:172.21.15.83:3300/0 conn(0x7fd7f4160770 0x7fd7f40b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.864+0000 7fd7f9382640 1 -- 172.21.15.79:0/289788726 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f4145210 con 0x7fd7f4160390 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.864+0000 7fd7f8b81640 1 --2- 172.21.15.79:0/289788726 >> v2:172.21.15.83:3300/0 conn(0x7fd7f4160770 0x7fd7f40b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.864+0000 7fd7f9b83640 1 --2- 172.21.15.79:0/289788726 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f41630d0 0x7fd7f4163560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.864+0000 7fd7f9382640 1 --2- 172.21.15.79:0/289788726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f4160390 0x7fd7f40b3b10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd7f406d780 tx=0x7fd7f002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb52737956f0618d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.864+0000 7fd7ebfff640 1 -- 172.21.15.79:0/289788726 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0036070 con 0x7fd7f4160390 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.864+0000 7fd7ebfff640 1 -- 172.21.15.79:0/289788726 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f0031600 con 0x7fd7f4160390 2024-04-26T19:51:00.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.864+0000 7fd7ebfff640 1 -- 172.21.15.79:0/289788726 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00319d0 con 0x7fd7f4160390 2024-04-26T19:51:00.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.865+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/289788726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f4160390 msgr2=0x7fd7f40b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.865+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/289788726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f4160390 0x7fd7f40b3b10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd7f406d780 tx=0x7fd7f002a610 comp rx=0 tx=0).stop 2024-04-26T19:51:00.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.865+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/289788726 shutdown_connections 2024-04-26T19:51:00.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.865+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/289788726 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f41630d0 0x7fd7f4163560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.865+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/289788726 >> v2:172.21.15.83:3300/0 conn(0x7fd7f4160770 0x7fd7f40b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.865+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/289788726 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f4160390 0x7fd7f40b3b10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.865+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/289788726 >> 172.21.15.79:0/289788726 conn(0x7fd7f40b2720 msgr2=0x7fd7f4070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:00.869 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.865+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/289788726 shutdown_connections 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.865+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/289788726 wait complete. 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.866+0000 7fd7fb5e6640 1 Processor -- start 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.866+0000 7fd7fb5e6640 1 -- start start 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7fb5e6640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f4160390 0x7fd7f40b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7fb5e6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f4160770 0x7fd7f406fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7fb5e6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f41630d0 0x7fd7f4070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7fb5e6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4145a10 con 0x7fd7f41630d0 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7fb5e6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f4163aa0 con 0x7fd7f4160770 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7fb5e6640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f4145b90 con 0x7fd7f4160390 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9b83640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f41630d0 0x7fd7f4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9b83640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f41630d0 0x7fd7f4070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46348/0 (socket says 172.21.15.79:46348) 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9b83640 1 -- 172.21.15.79:0/1835774285 learned_addr learned my addr 172.21.15.79:0/1835774285 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:00.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9382640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f4160390 0x7fd7f40b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f8b81640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f4160770 0x7fd7f406fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9b83640 1 -- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f4160390 msgr2=0x7fd7f40b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9b83640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f4160390 0x7fd7f40b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9b83640 1 -- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f4160770 msgr2=0x7fd7f406fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9b83640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f4160770 0x7fd7f406fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9b83640 1 -- 172.21.15.79:0/1835774285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f41f8d20 con 0x7fd7f41630d0 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9382640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f4160390 0x7fd7f40b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f8b81640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f4160770 0x7fd7f406fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.867+0000 7fd7f9b83640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f41630d0 0x7fd7f4070330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c820 tx=0x7fd7ec00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.868+0000 7fd7e9ffb640 1 -- 172.21.15.79:0/1835774285 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec015070 con 0x7fd7f41630d0 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.868+0000 7fd7e9ffb640 1 -- 172.21.15.79:0/1835774285 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ec007a30 con 0x7fd7f41630d0 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.868+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f41f9000 con 0x7fd7f41630d0 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.868+0000 7fd7e9ffb640 1 -- 172.21.15.79:0/1835774285 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ec007e20 con 0x7fd7f41630d0 2024-04-26T19:51:00.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.868+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f41f95c0 con 0x7fd7f41630d0 2024-04-26T19:51:00.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.869+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f40506c0 con 0x7fd7f41630d0 2024-04-26T19:51:00.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.869+0000 7fd7e9ffb640 1 -- 172.21.15.79:0/1835774285 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd7ec018070 con 0x7fd7f41630d0 2024-04-26T19:51:00.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.870+0000 7fd7e9ffb640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd7d806c920 0x7fd7d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:00.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.870+0000 7fd7e9ffb640 1 -- 172.21.15.79:0/1835774285 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd7ec08bd90 con 0x7fd7f41630d0 2024-04-26T19:51:00.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.870+0000 7fd7f9382640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd7d806c920 0x7fd7d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:00.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.870+0000 7fd7f9382640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd7d806c920 0x7fd7d806ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0033000 tx=0x7fd7f0009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:00.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:00.874+0000 7fd7e9ffb640 1 -- 172.21.15.79:0/1835774285 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd7ec059e80 con 0x7fd7f41630d0 2024-04-26T19:51:01.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.087+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7f40b2500 con 0x7fd7f41630d0 2024-04-26T19:51:01.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.088+0000 7fd7e9ffb640 1 -- 172.21.15.79:0/1835774285 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7fd7ec059780 con 0x7fd7f41630d0 2024-04-26T19:51:01.089 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:01.089 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:52.517190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:51:01.089 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2024-04-26T19:51:01.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd7d806c920 msgr2=0x7fd7d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd7d806c920 0x7fd7d806ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0033000 tx=0x7fd7f0009000 comp rx=0 tx=0).stop 2024-04-26T19:51:01.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f41630d0 msgr2=0x7fd7f4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f41630d0 0x7fd7f4070330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec00c820 tx=0x7fd7ec00cce0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 shutdown_connections 2024-04-26T19:51:01.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd7d806c920 0x7fd7d806ede0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd7f41630d0 0x7fd7f4070330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd7f4160770 0x7fd7f406fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 --2- 172.21.15.79:0/1835774285 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd7f4160390 0x7fd7f40b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 >> 172.21.15.79:0/1835774285 conn(0x7fd7f40b2720 msgr2=0x7fd7f415f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:01.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 shutdown_connections 2024-04-26T19:51:01.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.090+0000 7fd7fb5e6640 1 -- 172.21.15.79:0/1835774285 wait complete. 2024-04-26T19:51:01.102 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:01.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.198+0000 7efd30693640 1 Processor -- start 2024-04-26T19:51:01.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.198+0000 7efd30693640 1 -- start start 2024-04-26T19:51:01.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd30693640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efd2806fb20 0x7efd280d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd30693640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806ff00 0x7efd280d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd30693640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd281488c0 0x7efd28148d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd30693640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7efd280d3e60 con 0x7efd2806fb20 2024-04-26T19:51:01.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd30693640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efd280b1290 con 0x7efd281488c0 2024-04-26T19:51:01.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd30693640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7efd280d4090 con 0x7efd2806ff00 2024-04-26T19:51:01.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd2e42f640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efd2806fb20 0x7efd280d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd2e42f640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efd2806fb20 0x7efd280d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46364/0 (socket says 172.21.15.79:46364) 2024-04-26T19:51:01.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd2e42f640 1 -- 172.21.15.79:0/2544146931 learned_addr learned my addr 172.21.15.79:0/2544146931 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:01.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd2ec30640 1 --2- 172.21.15.79:0/2544146931 >> v2:172.21.15.83:3300/0 conn(0x7efd281488c0 0x7efd28148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd2dc2e640 1 --2- 172.21.15.79:0/2544146931 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806ff00 0x7efd280d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd2e42f640 1 -- 172.21.15.79:0/2544146931 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806ff00 msgr2=0x7efd280d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:01.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd2e42f640 1 --2- 172.21.15.79:0/2544146931 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806ff00 0x7efd280d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd2e42f640 1 -- 172.21.15.79:0/2544146931 >> v2:172.21.15.83:3300/0 conn(0x7efd281488c0 msgr2=0x7efd28148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:01.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.199+0000 7efd2e42f640 1 --2- 172.21.15.79:0/2544146931 >> v2:172.21.15.83:3300/0 conn(0x7efd281488c0 0x7efd28148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.200+0000 7efd2e42f640 1 -- 172.21.15.79:0/2544146931 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd280d4370 con 0x7efd2806fb20 2024-04-26T19:51:01.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.200+0000 7efd2e42f640 1 --2- 172.21.15.79:0/2544146931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd2806fb20 0x7efd280d2cd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd280b11e0 tx=0x7efd2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125609f0d3f58720 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:01.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.201+0000 7efd2d42d640 1 -- 172.21.15.79:0/2544146931 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24037070 con 0x7efd2806fb20 2024-04-26T19:51:01.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd2d42d640 1 -- 172.21.15.79:0/2544146931 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd24032440 con 0x7efd2806fb20 2024-04-26T19:51:01.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd2d42d640 1 -- 172.21.15.79:0/2544146931 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24032810 con 0x7efd2806fb20 2024-04-26T19:51:01.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd30693640 1 -- 172.21.15.79:0/2544146931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd2806fb20 msgr2=0x7efd280d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd30693640 1 --2- 172.21.15.79:0/2544146931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd2806fb20 0x7efd280d2cd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd280b11e0 tx=0x7efd2402a270 comp rx=0 tx=0).stop 2024-04-26T19:51:01.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd30693640 1 -- 172.21.15.79:0/2544146931 shutdown_connections 2024-04-26T19:51:01.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd30693640 1 --2- 172.21.15.79:0/2544146931 >> v2:172.21.15.83:3300/0 conn(0x7efd281488c0 0x7efd28148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd30693640 1 --2- 172.21.15.79:0/2544146931 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806ff00 0x7efd280d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd30693640 1 --2- 172.21.15.79:0/2544146931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd2806fb20 0x7efd280d2cd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd30693640 1 -- 172.21.15.79:0/2544146931 >> 172.21.15.79:0/2544146931 conn(0x7efd28071c30 msgr2=0x7efd2806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:01.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd30693640 1 -- 172.21.15.79:0/2544146931 shutdown_connections 2024-04-26T19:51:01.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.202+0000 7efd30693640 1 -- 172.21.15.79:0/2544146931 wait complete. 2024-04-26T19:51:01.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.203+0000 7efd30693640 1 Processor -- start 2024-04-26T19:51:01.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.203+0000 7efd30693640 1 -- start start 2024-04-26T19:51:01.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd30693640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806fb20 0x7efd2813f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd30693640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd2806ff00 0x7efd2815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd30693640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd281488c0 0x7efd280b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd30693640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efd280d4c10 con 0x7efd281488c0 2024-04-26T19:51:01.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd30693640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd280d4930 con 0x7efd2806ff00 2024-04-26T19:51:01.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd30693640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7efd28149270 con 0x7efd2806fb20 2024-04-26T19:51:01.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2ec30640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd281488c0 0x7efd280b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2ec30640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd281488c0 0x7efd280b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46370/0 (socket says 172.21.15.79:46370) 2024-04-26T19:51:01.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2ec30640 1 -- 172.21.15.79:0/1999920298 learned_addr learned my addr 172.21.15.79:0/1999920298 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:01.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2dc2e640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd2806ff00 0x7efd2815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2e42f640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806fb20 0x7efd2813f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2ec30640 1 -- 172.21.15.79:0/1999920298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806fb20 msgr2=0x7efd2813f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2ec30640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806fb20 0x7efd2813f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2ec30640 1 -- 172.21.15.79:0/1999920298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd2806ff00 msgr2=0x7efd2815eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2ec30640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd2806ff00 0x7efd2815eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2ec30640 1 -- 172.21.15.79:0/1999920298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd2815f0a0 con 0x7efd281488c0 2024-04-26T19:51:01.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.204+0000 7efd2e42f640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806fb20 0x7efd2813f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:01.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.205+0000 7efd2ec30640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd281488c0 0x7efd280b35f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd2000efc0 tx=0x7efd2000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.205+0000 7efd1effd640 1 -- 172.21.15.79:0/1999920298 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20018020 con 0x7efd281488c0 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.205+0000 7efd1effd640 1 -- 172.21.15.79:0/1999920298 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd20014480 con 0x7efd281488c0 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.205+0000 7efd1effd640 1 -- 172.21.15.79:0/1999920298 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd20014870 con 0x7efd281488c0 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.205+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd2813f690 con 0x7efd281488c0 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.205+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2813fc50 con 0x7efd281488c0 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.206+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd280506c0 con 0x7efd281488c0 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.206+0000 7efd1effd640 1 -- 172.21.15.79:0/1999920298 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd20005d10 con 0x7efd281488c0 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.206+0000 7efd1effd640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efd0c06c8f0 0x7efd0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.207+0000 7efd1effd640 1 -- 172.21.15.79:0/1999920298 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efd2008cc50 con 0x7efd281488c0 2024-04-26T19:51:01.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.207+0000 7efd2e42f640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efd0c06c8f0 0x7efd0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.207+0000 7efd2e42f640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efd0c06c8f0 0x7efd0c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efd24034000 tx=0x7efd2402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:01.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.211+0000 7efd1effd640 1 -- 172.21.15.79:0/1999920298 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd20092020 con 0x7efd281488c0 2024-04-26T19:51:01.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.387+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd28073aa0 con 0x7efd281488c0 2024-04-26T19:51:01.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.389+0000 7efd1effd640 1 -- 172.21.15.79:0/1999920298 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7efd20022030 con 0x7efd281488c0 2024-04-26T19:51:01.389 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:01.390 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efd0c06c8f0 msgr2=0x7efd0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efd0c06c8f0 0x7efd0c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efd24034000 tx=0x7efd2402e000 comp rx=0 tx=0).stop 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd281488c0 msgr2=0x7efd280b35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd281488c0 0x7efd280b35f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efd2000efc0 tx=0x7efd2000c490 comp rx=0 tx=0).stop 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 shutdown_connections 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efd0c06c8f0 0x7efd0c06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd281488c0 0x7efd280b35f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd2806ff00 0x7efd2815eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 --2- 172.21.15.79:0/1999920298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd2806fb20 0x7efd2813f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 >> 172.21.15.79:0/1999920298 conn(0x7efd28071c30 msgr2=0x7efd2813a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:01.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 shutdown_connections 2024-04-26T19:51:01.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.391+0000 7efd30693640 1 -- 172.21.15.79:0/1999920298 wait complete. 2024-04-26T19:51:01.403 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:01.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:01.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.502+0000 7f936ae59640 1 Processor -- start 2024-04-26T19:51:01.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.502+0000 7f936ae59640 1 -- start start 2024-04-26T19:51:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f936ae59640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f93640aeeb0 0x7f936406f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f936ae59640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9364070160 0x7f93640b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f936ae59640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640b1990 0x7f93640af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f936ae59640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f936406fa10 con 0x7f93640aeeb0 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f936ae59640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9364072ea0 con 0x7f9364070160 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f936ae59640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9364073180 con 0x7f93640b1990 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f9368bf5640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f93640aeeb0 0x7f936406f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f9368bf5640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f93640aeeb0 0x7f936406f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46386/0 (socket says 172.21.15.79:46386) 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f9368bf5640 1 -- 172.21.15.79:0/3022368012 learned_addr learned my addr 172.21.15.79:0/3022368012 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f93693f6640 1 --2- 172.21.15.79:0/3022368012 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640b1990 0x7f93640af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.503+0000 7f9363fff640 1 --2- 172.21.15.79:0/3022368012 >> v2:172.21.15.83:3300/0 conn(0x7f9364070160 0x7f93640b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.504+0000 7f9368bf5640 1 -- 172.21.15.79:0/3022368012 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640b1990 msgr2=0x7f93640af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.504+0000 7f9368bf5640 1 --2- 172.21.15.79:0/3022368012 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640b1990 0x7f93640af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.504+0000 7f9368bf5640 1 -- 172.21.15.79:0/3022368012 >> v2:172.21.15.83:3300/0 conn(0x7f9364070160 msgr2=0x7f93640b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.504+0000 7f9368bf5640 1 --2- 172.21.15.79:0/3022368012 >> v2:172.21.15.83:3300/0 conn(0x7f9364070160 0x7f93640b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.504+0000 7f9368bf5640 1 -- 172.21.15.79:0/3022368012 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9364073460 con 0x7f93640aeeb0 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.504+0000 7f9368bf5640 1 --2- 172.21.15.79:0/3022368012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93640aeeb0 0x7f936406f0e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f93640aee00 tx=0x7f935402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a7bb7bbf6d01d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.504+0000 7f93637fe640 1 -- 172.21.15.79:0/3022368012 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354037070 con 0x7f93640aeeb0 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.504+0000 7f93637fe640 1 -- 172.21.15.79:0/3022368012 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9354032440 con 0x7f93640aeeb0 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.504+0000 7f93637fe640 1 -- 172.21.15.79:0/3022368012 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354032810 con 0x7f93640aeeb0 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.505+0000 7f936ae59640 1 -- 172.21.15.79:0/3022368012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93640aeeb0 msgr2=0x7f936406f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.505+0000 7f936ae59640 1 --2- 172.21.15.79:0/3022368012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93640aeeb0 0x7f936406f0e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f93640aee00 tx=0x7f935402a140 comp rx=0 tx=0).stop 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.505+0000 7f936ae59640 1 -- 172.21.15.79:0/3022368012 shutdown_connections 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.505+0000 7f936ae59640 1 --2- 172.21.15.79:0/3022368012 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640b1990 0x7f93640af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.505+0000 7f936ae59640 1 --2- 172.21.15.79:0/3022368012 >> v2:172.21.15.83:3300/0 conn(0x7f9364070160 0x7f93640b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.505+0000 7f936ae59640 1 --2- 172.21.15.79:0/3022368012 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93640aeeb0 0x7f936406f0e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.505+0000 7f936ae59640 1 -- 172.21.15.79:0/3022368012 >> 172.21.15.79:0/3022368012 conn(0x7f93640687e0 msgr2=0x7f9364071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.505+0000 7f936ae59640 1 -- 172.21.15.79:0/3022368012 shutdown_connections 2024-04-26T19:51:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.505+0000 7f936ae59640 1 -- 172.21.15.79:0/3022368012 wait complete. 2024-04-26T19:51:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.506+0000 7f936ae59640 1 Processor -- start 2024-04-26T19:51:01.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.506+0000 7f936ae59640 1 -- start start 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f936ae59640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9364070160 0x7f93640c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f936ae59640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640aeeb0 0x7f936413f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f936ae59640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93640b1990 0x7f936414fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f936ae59640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f936406f620 con 0x7f9364070160 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f936ae59640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9364149380 con 0x7f93640b1990 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f936ae59640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f936406f7a0 con 0x7f93640aeeb0 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f9368bf5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9364070160 0x7f93640c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f9368bf5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9364070160 0x7f93640c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46390/0 (socket says 172.21.15.79:46390) 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f9368bf5640 1 -- 172.21.15.79:0/1977290289 learned_addr learned my addr 172.21.15.79:0/1977290289 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f93693f6640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93640b1990 0x7f936414fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f9363fff640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640aeeb0 0x7f936413f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f9368bf5640 1 -- 172.21.15.79:0/1977290289 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640aeeb0 msgr2=0x7f936413f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f9368bf5640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640aeeb0 0x7f936413f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f9368bf5640 1 -- 172.21.15.79:0/1977290289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93640b1990 msgr2=0x7f936414fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f9368bf5640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93640b1990 0x7f936414fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f9368bf5640 1 -- 172.21.15.79:0/1977290289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9364150340 con 0x7f9364070160 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.507+0000 7f93693f6640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93640b1990 0x7f936414fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.508+0000 7f9368bf5640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9364070160 0x7f93640c4e70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f935402a180 tx=0x7f93540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.508+0000 7f93617fa640 1 -- 172.21.15.79:0/1977290289 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354041020 con 0x7f9364070160 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.508+0000 7f93617fa640 1 -- 172.21.15.79:0/1977290289 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93540306c0 con 0x7f9364070160 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.508+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9364150610 con 0x7f9364070160 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.508+0000 7f93617fa640 1 -- 172.21.15.79:0/1977290289 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9354037040 con 0x7f9364070160 2024-04-26T19:51:01.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.508+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9364150bd0 con 0x7f9364070160 2024-04-26T19:51:01.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.509+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93640506c0 con 0x7f9364070160 2024-04-26T19:51:01.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.509+0000 7f93617fa640 1 -- 172.21.15.79:0/1977290289 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9354044020 con 0x7f9364070160 2024-04-26T19:51:01.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.510+0000 7f93617fa640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f935006c8f0 0x7f935006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.510+0000 7f93617fa640 1 -- 172.21.15.79:0/1977290289 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f93540ac5c0 con 0x7f9364070160 2024-04-26T19:51:01.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.510+0000 7f9363fff640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f935006c8f0 0x7f935006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.510+0000 7f9363fff640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f935006c8f0 0x7f935006edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9358006fd0 tx=0x7f9358008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:01.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.514+0000 7f93617fa640 1 -- 172.21.15.79:0/1977290289 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f935404b030 con 0x7f9364070160 2024-04-26T19:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.688+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f936406a800 con 0x7f9364070160 2024-04-26T19:51:01.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.689+0000 7f93617fa640 1 -- 172.21.15.79:0/1977290289 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f935407a740 con 0x7f9364070160 2024-04-26T19:51:01.689 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.690+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f935006c8f0 msgr2=0x7f935006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f935006c8f0 0x7f935006edb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9358006fd0 tx=0x7f9358008040 comp rx=0 tx=0).stop 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9364070160 msgr2=0x7f93640c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9364070160 0x7f93640c4e70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f935402a180 tx=0x7f93540046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 shutdown_connections 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f935006c8f0 0x7f935006edb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93640b1990 0x7f936414fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93640aeeb0 0x7f936413f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 --2- 172.21.15.79:0/1977290289 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9364070160 0x7f93640c4e70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 >> 172.21.15.79:0/1977290289 conn(0x7f93640687e0 msgr2=0x7f936406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:01.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 shutdown_connections 2024-04-26T19:51:01.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.691+0000 7f936ae59640 1 -- 172.21.15.79:0/1977290289 wait complete. 2024-04-26T19:51:01.702 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:01.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.801+0000 7f07c2b9d640 1 Processor -- start 2024-04-26T19:51:01.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.801+0000 7f07c2b9d640 1 -- start start 2024-04-26T19:51:01.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.802+0000 7f07c2b9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f07bc146bd0 0x7f07bc161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.802+0000 7f07c2b9d640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07bc161bd0 con 0x7f07bc06df80 2024-04-26T19:51:01.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.802+0000 7f07c2b9d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07bc06e540 con 0x7f07bc074130 2024-04-26T19:51:01.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.802+0000 7f07c2b9d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f07bc06e820 con 0x7f07bc146bd0 2024-04-26T19:51:01.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.802+0000 7f07bbfff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f07bc06df80 0x7f07bc073a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53132/0 (socket says 172.21.15.79:53132) 2024-04-26T19:51:01.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.802+0000 7f07bbfff640 1 -- 172.21.15.79:0/855795692 learned_addr learned my addr 172.21.15.79:0/855795692 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:01.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.802+0000 7f07c0939640 1 --2- 172.21.15.79:0/855795692 >> v2:172.21.15.83:3300/0 conn(0x7f07bc146bd0 0x7f07bc161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.803+0000 7f07bb7fe640 1 -- 172.21.15.79:0/855795692 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3842527755 0 0) 0x7f07bc161bd0 con 0x7f07bc06df80 2024-04-26T19:51:01.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.803+0000 7f07bb7fe640 1 -- 172.21.15.79:0/855795692 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07a4003520 con 0x7f07bc06df80 2024-04-26T19:51:01.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.803+0000 7f07c0939640 1 -- 172.21.15.79:0/855795692 >> v1:172.21.15.83:6789/0 conn(0x7f07bc074130 legacy=0x7f07bc15ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.803+0000 7f07c0939640 1 -- 172.21.15.79:0/855795692 >> v1:172.21.15.79:6789/0 conn(0x7f07bc06df80 legacy=0x7f07bc073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.803+0000 7f07c0939640 1 -- 172.21.15.79:0/855795692 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc06eb00 con 0x7f07bc146bd0 2024-04-26T19:51:01.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.803+0000 7f07c0939640 1 --2- 172.21.15.79:0/855795692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc146bd0 0x7f07bc161690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0097a0 tx=0x7f07ac02a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d4831cfe85c35d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07bb7fe640 1 -- 172.21.15.79:0/855795692 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac037070 con 0x7f07bc146bd0 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07bb7fe640 1 -- 172.21.15.79:0/855795692 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ac032440 con 0x7f07bc146bd0 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07bb7fe640 1 -- 172.21.15.79:0/855795692 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac0327f0 con 0x7f07bc146bd0 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07c2b9d640 1 -- 172.21.15.79:0/855795692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc146bd0 msgr2=0x7f07bc161690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07c2b9d640 1 --2- 172.21.15.79:0/855795692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc146bd0 0x7f07bc161690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0097a0 tx=0x7f07ac02a230 comp rx=0 tx=0).stop 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07c2b9d640 1 -- 172.21.15.79:0/855795692 shutdown_connections 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07c2b9d640 1 --2- 172.21.15.79:0/855795692 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc146bd0 0x7f07bc161690 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07c2b9d640 1 -- 172.21.15.79:0/855795692 >> 172.21.15.79:0/855795692 conn(0x7f07bc0b2540 msgr2=0x7f07bc0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07c2b9d640 1 -- 172.21.15.79:0/855795692 shutdown_connections 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.804+0000 7f07c2b9d640 1 -- 172.21.15.79:0/855795692 wait complete. 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.805+0000 7f07c2b9d640 1 Processor -- start 2024-04-26T19:51:01.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.805+0000 7f07c2b9d640 1 -- start start 2024-04-26T19:51:01.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c2b9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07bc06df80 0x7f07bc1f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c2b9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc074130 0x7f07bc1f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c2b9d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07bc146bd0 0x7f07bc1f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c2b9d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc15f480 con 0x7f07bc06df80 2024-04-26T19:51:01.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c2b9d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f07bc15e850 con 0x7f07bc074130 2024-04-26T19:51:01.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c2b9d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f07bc15f600 con 0x7f07bc146bd0 2024-04-26T19:51:01.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c0939640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07bc06df80 0x7f07bc1f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c0939640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07bc06df80 0x7f07bc1f3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46394/0 (socket says 172.21.15.79:46394) 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c0939640 1 -- 172.21.15.79:0/687120579 learned_addr learned my addr 172.21.15.79:0/687120579 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07c113a640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07bc146bd0 0x7f07bc1f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.806+0000 7f07bbfff640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc074130 0x7f07bc1f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07c0939640 1 -- 172.21.15.79:0/687120579 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07bc146bd0 msgr2=0x7f07bc1f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07c0939640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07bc146bd0 0x7f07bc1f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07c0939640 1 -- 172.21.15.79:0/687120579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc074130 msgr2=0x7f07bc1f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07c0939640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc074130 0x7f07bc1f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07c0939640 1 -- 172.21.15.79:0/687120579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07bc1f3990 con 0x7f07bc06df80 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07c0939640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07bc06df80 0x7f07bc1f3450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0097a0 tx=0x7f07ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07b97fa640 1 -- 172.21.15.79:0/687120579 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac037050 con 0x7f07bc06df80 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07b97fa640 1 -- 172.21.15.79:0/687120579 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07ac038040 con 0x7f07bc06df80 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07bc0aaa80 con 0x7f07bc06df80 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07b97fa640 1 -- 172.21.15.79:0/687120579 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac004470 con 0x7f07bc06df80 2024-04-26T19:51:01.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.807+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07bc1339e0 con 0x7f07bc06df80 2024-04-26T19:51:01.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.808+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07bc0aac50 con 0x7f07bc06df80 2024-04-26T19:51:01.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.808+0000 7f07b97fa640 1 -- 172.21.15.79:0/687120579 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f07ac03c6d0 con 0x7f07bc06df80 2024-04-26T19:51:01.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.809+0000 7f07b97fa640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f07a806c8f0 0x7f07a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:01.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.809+0000 7f07b97fa640 1 -- 172.21.15.79:0/687120579 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f07ac0ac6e0 con 0x7f07bc06df80 2024-04-26T19:51:01.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.809+0000 7f07bbfff640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f07a806c8f0 0x7f07a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:01.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.809+0000 7f07bbfff640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f07a806c8f0 0x7f07a806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0d0380 tx=0x7f07b0007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:01.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.813+0000 7f07b97fa640 1 -- 172.21.15.79:0/687120579 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f07ac07a210 con 0x7f07bc06df80 2024-04-26T19:51:01.988 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.988+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f07bc06f5e0 con 0x7f07bc06df80 2024-04-26T19:51:01.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.988+0000 7f07b97fa640 1 -- 172.21.15.79:0/687120579 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f07ac07e510 con 0x7f07bc06df80 2024-04-26T19:51:01.989 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:01.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f07a806c8f0 msgr2=0x7f07a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f07a806c8f0 0x7f07a806edb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f07bc0d0380 tx=0x7f07b0007d30 comp rx=0 tx=0).stop 2024-04-26T19:51:01.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07bc06df80 msgr2=0x7f07bc1f3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:01.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07bc06df80 0x7f07bc1f3450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0097a0 tx=0x7f07ac0046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 shutdown_connections 2024-04-26T19:51:01.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f07a806c8f0 0x7f07a806edb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f07bc146bd0 0x7f07bc1f3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f07bc074130 0x7f07bc1f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 --2- 172.21.15.79:0/687120579 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f07bc06df80 0x7f07bc1f3450 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:01.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 >> 172.21.15.79:0/687120579 conn(0x7f07bc0b2540 msgr2=0x7f07bc0b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:01.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 shutdown_connections 2024-04-26T19:51:01.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:01.990+0000 7f07c2b9d640 1 -- 172.21.15.79:0/687120579 wait complete. 2024-04-26T19:51:02.002 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:02.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.101+0000 7f1cf535f640 1 Processor -- start 2024-04-26T19:51:02.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.101+0000 7f1cf535f640 1 -- start start 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1cf535f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf015c130 0x7f1cf00d1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1cf535f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1cf00b73c0 0x7f1cf00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1cf535f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cf015fd90 con 0x7f1cf00cfc20 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1cf535f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1cf00b7d60 con 0x7f1cf00b73c0 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1cf535f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf0160070 con 0x7f1cf015c130 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1ceeffd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf015c130 0x7f1cf00d1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1cee7fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1cf00b73c0 0x7f1cf00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1cef7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1cf00cfc20 0x7f1cf00d0030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:49308/0 (socket says 172.21.15.79:49308) 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1cef7fe640 1 -- 172.21.15.79:0/2831532360 learned_addr learned my addr 172.21.15.79:0/2831532360 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1ceeffd640 1 -- 172.21.15.79:0/2831532360 >> v2:172.21.15.83:3300/0 conn(0x7f1cf00b73c0 msgr2=0x7f1cf00b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:02.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1ceeffd640 1 --2- 172.21.15.79:0/2831532360 >> v2:172.21.15.83:3300/0 conn(0x7f1cf00b73c0 0x7f1cf00b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1ceeffd640 1 -- 172.21.15.79:0/2831532360 >> v1:172.21.15.83:6789/0 conn(0x7f1cf00cfc20 legacy=0x7f1cf00d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1ceeffd640 1 -- 172.21.15.79:0/2831532360 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf0160350 con 0x7f1cf015c130 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.102+0000 7f1cee7fc640 1 --2- 172.21.15.79:0/2831532360 >> v2:172.21.15.83:3300/0 conn(0x7f1cf00b73c0 0x7f1cf00b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.103+0000 7f1ceeffd640 1 --2- 172.21.15.79:0/2831532360 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf015c130 0x7f1cf00d1350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80097a0 tx=0x7f1ce802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48066c661b54d7f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.103+0000 7f1cedffb640 1 -- 172.21.15.79:0/2831532360 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8037070 con 0x7f1cf015c130 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.103+0000 7f1cedffb640 1 -- 172.21.15.79:0/2831532360 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce802ae50 con 0x7f1cf015c130 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.103+0000 7f1cedffb640 1 -- 172.21.15.79:0/2831532360 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce8032660 con 0x7f1cf015c130 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.104+0000 7f1cf535f640 1 -- 172.21.15.79:0/2831532360 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf015c130 msgr2=0x7f1cf00d1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.104+0000 7f1cf535f640 1 --2- 172.21.15.79:0/2831532360 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf015c130 0x7f1cf00d1350 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1ce80097a0 tx=0x7f1ce802a100 comp rx=0 tx=0).stop 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.104+0000 7f1cf535f640 1 -- 172.21.15.79:0/2831532360 shutdown_connections 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.104+0000 7f1cf535f640 1 --2- 172.21.15.79:0/2831532360 >> v2:172.21.15.83:3300/0 conn(0x7f1cf00b73c0 0x7f1cf00b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.104+0000 7f1cf535f640 1 --2- 172.21.15.79:0/2831532360 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf015c130 0x7f1cf00d1350 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.104+0000 7f1cf535f640 1 -- 172.21.15.79:0/2831532360 >> 172.21.15.79:0/2831532360 conn(0x7f1cf00b4360 msgr2=0x7f1cf00b4740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:02.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.104+0000 7f1cf535f640 1 -- 172.21.15.79:0/2831532360 shutdown_connections 2024-04-26T19:51:02.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.104+0000 7f1cf535f640 1 -- 172.21.15.79:0/2831532360 wait complete. 2024-04-26T19:51:02.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.105+0000 7f1cf535f640 1 Processor -- start 2024-04-26T19:51:02.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.105+0000 7f1cf535f640 1 -- start start 2024-04-26T19:51:02.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cf535f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf00b73c0 0x7f1cf0146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cf535f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf00cfc20 0x7f1cf014a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cf535f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cf015c130 0x7f1cf014f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cf535f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf015c690 con 0x7f1cf00cfc20 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cf535f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1cf015c510 con 0x7f1cf015c130 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cf535f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1cf015d590 con 0x7f1cf00b73c0 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cee7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf00cfc20 0x7f1cf014a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cee7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf00cfc20 0x7f1cf014a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46404/0 (socket says 172.21.15.79:46404) 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1ceeffd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf00b73c0 0x7f1cf0146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cee7fc640 1 -- 172.21.15.79:0/1920334770 learned_addr learned my addr 172.21.15.79:0/1920334770 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.106+0000 7f1cef7fe640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cf015c130 0x7f1cf014f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cee7fc640 1 -- 172.21.15.79:0/1920334770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf00b73c0 msgr2=0x7f1cf0146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cee7fc640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf00b73c0 0x7f1cf0146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cee7fc640 1 -- 172.21.15.79:0/1920334770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cf015c130 msgr2=0x7f1cf014f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cee7fc640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cf015c130 0x7f1cf014f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cee7fc640 1 -- 172.21.15.79:0/1920334770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cf0149900 con 0x7f1cf00cfc20 2024-04-26T19:51:02.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cee7fc640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf00cfc20 0x7f1cf014a090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400efc0 tx=0x7f1ce400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:02.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cd77fe640 1 -- 172.21.15.79:0/1920334770 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4018020 con 0x7f1cf00cfc20 2024-04-26T19:51:02.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cd77fe640 1 -- 172.21.15.79:0/1920334770 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce4014680 con 0x7f1cf00cfc20 2024-04-26T19:51:02.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cf0151470 con 0x7f1cf00cfc20 2024-04-26T19:51:02.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cd77fe640 1 -- 172.21.15.79:0/1920334770 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce4014a50 con 0x7f1cf00cfc20 2024-04-26T19:51:02.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.107+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cf014fd90 con 0x7f1cf00cfc20 2024-04-26T19:51:02.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.108+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf00506c0 con 0x7f1cf00cfc20 2024-04-26T19:51:02.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.109+0000 7f1cd77fe640 1 -- 172.21.15.79:0/1920334770 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1ce4005d10 con 0x7f1cf00cfc20 2024-04-26T19:51:02.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.109+0000 7f1cd77fe640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.109+0000 7f1cd77fe640 1 -- 172.21.15.79:0/1920334770 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ce408cb70 con 0x7f1cf00cfc20 2024-04-26T19:51:02.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.109+0000 7f1ceeffd640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.109+0000 7f1ceeffd640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8034000 tx=0x7f1ce802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:02.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.114+0000 7f1cd77fe640 1 -- 172.21.15.79:0/1920334770 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1ce4022030 con 0x7f1cf00cfc20 2024-04-26T19:51:02.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.291+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1cf0070640 con 0x7f1cf00cfc20 2024-04-26T19:51:02.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.291+0000 7f1cd77fe640 1 -- 172.21.15.79:0/1920334770 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f1ce4010040 con 0x7f1cf00cfc20 2024-04-26T19:51:02.292 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1cdc06c8f0 msgr2=0x7f1cdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8034000 tx=0x7f1ce802e000 comp rx=0 tx=0).stop 2024-04-26T19:51:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf00cfc20 msgr2=0x7f1cf014a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf00cfc20 0x7f1cf014a090 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ce400efc0 tx=0x7f1ce400c490 comp rx=0 tx=0).stop 2024-04-26T19:51:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 shutdown_connections 2024-04-26T19:51:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1cdc06c8f0 0x7f1cdc06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1cf015c130 0x7f1cf014f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1cf00cfc20 0x7f1cf014a090 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 --2- 172.21.15.79:0/1920334770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1cf00b73c0 0x7f1cf0146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 >> 172.21.15.79:0/1920334770 conn(0x7f1cf00b4360 msgr2=0x7f1cf0155c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:02.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 shutdown_connections 2024-04-26T19:51:02.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.293+0000 7f1cf535f640 1 -- 172.21.15.79:0/1920334770 wait complete. 2024-04-26T19:51:02.305 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:02.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.404+0000 7f0d15504640 1 Processor -- start 2024-04-26T19:51:02.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.404+0000 7f0d15504640 1 -- start start 2024-04-26T19:51:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.404+0000 7f0d15504640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d10070160 0x7f0d100b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.404+0000 7f0d15504640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d1006fa10 con 0x7f0d100aeeb0 2024-04-26T19:51:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.404+0000 7f0d15504640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d10072ea0 con 0x7f0d100b1990 2024-04-26T19:51:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.404+0000 7f0d15504640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1006f7f0 con 0x7f0d10070160 2024-04-26T19:51:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0effd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0d100aeeb0 0x7f0d1006f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53144/0 (socket says 172.21.15.79:53144) 2024-04-26T19:51:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0effd640 1 -- 172.21.15.79:0/1851921114 learned_addr learned my addr 172.21.15.79:0/1851921114 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d07fff640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d10070160 0x7f0d100b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3839735260 0 0) 0x7f0d1006fa10 con 0x7f0d100aeeb0 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cfc003520 con 0x7f0d100aeeb0 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4104897514 0 0) 0x7f0cfc003520 con 0x7f0d100aeeb0 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d10070160 msgr2=0x7f0d100b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d10070160 0x7f0d100b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 >> v1:172.21.15.83:6789/0 conn(0x7f0d100b1990 legacy=0x7f0d100af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d10073180 con 0x7f0d100aeeb0 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f0d00001670 con 0x7f0d100aeeb0 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 >> v1:172.21.15.79:6789/0 conn(0x7f0d100aeeb0 legacy=0x7f0d1006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d10070160 0x7f0cfc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc003f60 0x7f0cfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.412 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc004d40 0x7f0cfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1006f7f0 con 0x7f0cfc003f60 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc00bb50 con 0x7f0cfc004d40 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0cfc00bd50 con 0x7f0d10070160 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d07fff640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc003f60 0x7f0cfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0effd640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d10070160 0x7f0cfc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.405+0000 7f0d0f7fe640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc004d40 0x7f0cfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.406+0000 7f0d07fff640 1 -- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d10070160 msgr2=0x7f0cfc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.406+0000 7f0d07fff640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d10070160 0x7f0cfc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.406+0000 7f0d07fff640 1 -- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc004d40 msgr2=0x7f0cfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.406+0000 7f0d07fff640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0cfc004d40 0x7f0cfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.413 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.406+0000 7f0d07fff640 1 -- 172.21.15.79:0/1851921114 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc003520 con 0x7f0cfc003f60 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.406+0000 7f0d07fff640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc003f60 0x7f0cfc004800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009a90 tx=0x7f0cfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0883cefbe932fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.406+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8036070 con 0x7f0cfc003f60 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.406+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8031400 con 0x7f0cfc003f60 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.406+0000 7f0d0e7fc640 1 -- 172.21.15.79:0/1851921114 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8009690 con 0x7f0cfc003f60 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.407+0000 7f0d15504640 1 -- 172.21.15.79:0/1851921114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc003f60 msgr2=0x7f0cfc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.407+0000 7f0d15504640 1 --2- 172.21.15.79:0/1851921114 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0cfc003f60 0x7f0cfc004800 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009a90 tx=0x7f0cfc005830 comp rx=0 tx=0).stop 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.407+0000 7f0d0f7fe640 1 -- 172.21.15.79:0/1851921114 reap_dead start 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.407+0000 7f0d15504640 1 -- 172.21.15.79:0/1851921114 shutdown_connections 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.407+0000 7f0d15504640 1 -- 172.21.15.79:0/1851921114 >> 172.21.15.79:0/1851921114 conn(0x7f0d100687e0 msgr2=0x7f0d10071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.407+0000 7f0d15504640 1 -- 172.21.15.79:0/1851921114 shutdown_connections 2024-04-26T19:51:02.414 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.407+0000 7f0d15504640 1 -- 172.21.15.79:0/1851921114 wait complete. 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.408+0000 7f0d15504640 1 Processor -- start 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.408+0000 7f0d15504640 1 -- start start 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d15504640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0cfc003f60 0x7f0d1015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d15504640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1015cc90 0x7f0d1012f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d15504640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1012fe80 0x7f0d10130330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d15504640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d10149380 con 0x7f0d1015cc90 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d15504640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d10073740 con 0x7f0d1012fe80 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d15504640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0d100af9a0 con 0x7f0cfc003f60 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1015cc90 0x7f0d1012f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1015cc90 0x7f0d1012f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46418/0 (socket says 172.21.15.79:46418) 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d07fff640 1 -- 172.21.15.79:0/3248163978 learned_addr learned my addr 172.21.15.79:0/3248163978 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:02.415 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d0f7fe640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1012fe80 0x7f0d10130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d0effd640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0cfc003f60 0x7f0d1015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d07fff640 1 -- 172.21.15.79:0/3248163978 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0cfc003f60 msgr2=0x7f0d1015c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d07fff640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0cfc003f60 0x7f0d1015c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d07fff640 1 -- 172.21.15.79:0/3248163978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1012fe80 msgr2=0x7f0d10130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d07fff640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1012fe80 0x7f0d10130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d07fff640 1 -- 172.21.15.79:0/3248163978 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1015e240 con 0x7f0d1015cc90 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d0f7fe640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1012fe80 0x7f0d10130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.409+0000 7f0d07fff640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1015cc90 0x7f0d1012f940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0cf802ab70 tx=0x7f0cf8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.410+0000 7f0d077fe640 1 -- 172.21.15.79:0/3248163978 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8040020 con 0x7f0d1015cc90 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.410+0000 7f0d077fe640 1 -- 172.21.15.79:0/3248163978 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cf8003bc0 con 0x7f0d1015cc90 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.410+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1015d1a0 con 0x7f0d1015cc90 2024-04-26T19:51:02.416 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.410+0000 7f0d077fe640 1 -- 172.21.15.79:0/3248163978 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8036040 con 0x7f0d1015cc90 2024-04-26T19:51:02.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.410+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1015d760 con 0x7f0d1015cc90 2024-04-26T19:51:02.417 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.411+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d100506c0 con 0x7f0d1015cc90 2024-04-26T19:51:02.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.411+0000 7f0d077fe640 1 -- 172.21.15.79:0/3248163978 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cf8043020 con 0x7f0d1015cc90 2024-04-26T19:51:02.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.412+0000 7f0d077fe640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf006c8f0 0x7f0cf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.412+0000 7f0d0effd640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf006c8f0 0x7f0cf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.412+0000 7f0d077fe640 1 -- 172.21.15.79:0/3248163978 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0cf80ac160 con 0x7f0d1015cc90 2024-04-26T19:51:02.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.412+0000 7f0d0effd640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf006c8f0 0x7f0cf006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d00007d10 tx=0x7f0d000078f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:02.418 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.416+0000 7f0d077fe640 1 -- 172.21.15.79:0/3248163978 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0cf804a030 con 0x7f0d1015cc90 2024-04-26T19:51:02.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.591+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0d1006a800 con 0x7f0d1015cc90 2024-04-26T19:51:02.592 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.591+0000 7f0d077fe640 1 -- 172.21.15.79:0/3248163978 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f0cf80044e0 con 0x7f0d1015cc90 2024-04-26T19:51:02.592 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:02.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf006c8f0 msgr2=0x7f0cf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf006c8f0 0x7f0cf006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d00007d10 tx=0x7f0d000078f0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1015cc90 msgr2=0x7f0d1012f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1015cc90 0x7f0d1012f940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0cf802ab70 tx=0x7f0cf8004730 comp rx=0 tx=0).stop 2024-04-26T19:51:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 shutdown_connections 2024-04-26T19:51:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf006c8f0 0x7f0cf006edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1012fe80 0x7f0d10130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1015cc90 0x7f0d1012f940 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 --2- 172.21.15.79:0/3248163978 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0cfc003f60 0x7f0d1015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.593+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 >> 172.21.15.79:0/3248163978 conn(0x7f0d100687e0 msgr2=0x7f0d100d4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.594+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 shutdown_connections 2024-04-26T19:51:02.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.594+0000 7f0d15504640 1 -- 172.21.15.79:0/3248163978 wait complete. 2024-04-26T19:51:02.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:51:02.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.701+0000 7fa31e26b640 1 Processor -- start 2024-04-26T19:51:02.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.701+0000 7fa31e26b640 1 -- start start 2024-04-26T19:51:02.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.702+0000 7fa31e26b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa318163060 0x7fa3181604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.702+0000 7fa31e26b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa3180708a0 0x7fa3180d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.702+0000 7fa31e26b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3181634f0 con 0x7fa3180704c0 2024-04-26T19:51:02.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.702+0000 7fa31e26b640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fa318142840 con 0x7fa3180708a0 2024-04-26T19:51:02.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.702+0000 7fa31e26b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa3181637d0 con 0x7fa318163060 2024-04-26T19:51:02.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.702+0000 7fa31c808640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa3180708a0 0x7fa3180d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.702+0000 7fa31c808640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa3180708a0 0x7fa3180d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46432/0 (socket says 172.21.15.79:46432) 2024-04-26T19:51:02.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.702+0000 7fa31c808640 1 -- 172.21.15.79:0/1801688594 learned_addr learned my addr 172.21.15.79:0/1801688594 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:02.704 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.703+0000 7fa3177fe640 1 --2- 172.21.15.79:0/1801688594 >> v2:172.21.15.83:3300/0 conn(0x7fa318163060 0x7fa3181604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.703+0000 7fa31c808640 1 -- 172.21.15.79:0/1801688594 >> v2:172.21.15.83:3300/0 conn(0x7fa318163060 msgr2=0x7fa3181604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.703+0000 7fa31c808640 1 --2- 172.21.15.79:0/1801688594 >> v2:172.21.15.83:3300/0 conn(0x7fa318163060 0x7fa3181604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.703+0000 7fa31c808640 1 -- 172.21.15.79:0/1801688594 >> v1:172.21.15.83:6789/0 conn(0x7fa3180704c0 legacy=0x7fa318142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.703+0000 7fa31c808640 1 -- 172.21.15.79:0/1801688594 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa318163ab0 con 0x7fa3180708a0 2024-04-26T19:51:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.703+0000 7fa31c808640 1 --2- 172.21.15.79:0/1801688594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180708a0 0x7fa3180d1430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa3180d20a0 tx=0x7fa31002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9877eafcefebcc12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.703+0000 7fa316ffd640 1 -- 172.21.15.79:0/1801688594 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310036070 con 0x7fa3180708a0 2024-04-26T19:51:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa316ffd640 1 -- 172.21.15.79:0/1801688594 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa310031600 con 0x7fa3180708a0 2024-04-26T19:51:02.705 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa316ffd640 1 -- 172.21.15.79:0/1801688594 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100319d0 con 0x7fa3180708a0 2024-04-26T19:51:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa31e26b640 1 -- 172.21.15.79:0/1801688594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180708a0 msgr2=0x7fa3180d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa31e26b640 1 --2- 172.21.15.79:0/1801688594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180708a0 0x7fa3180d1430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa3180d20a0 tx=0x7fa31002a6f0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa31e26b640 1 -- 172.21.15.79:0/1801688594 shutdown_connections 2024-04-26T19:51:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa31e26b640 1 --2- 172.21.15.79:0/1801688594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180708a0 0x7fa3180d1430 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa31e26b640 1 --2- 172.21.15.79:0/1801688594 >> v2:172.21.15.83:3300/0 conn(0x7fa318163060 0x7fa3181604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa31e26b640 1 -- 172.21.15.79:0/1801688594 >> 172.21.15.79:0/1801688594 conn(0x7fa318073820 msgr2=0x7fa3180d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:02.706 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa31e26b640 1 -- 172.21.15.79:0/1801688594 shutdown_connections 2024-04-26T19:51:02.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.704+0000 7fa31e26b640 1 -- 172.21.15.79:0/1801688594 wait complete. 2024-04-26T19:51:02.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.705+0000 7fa31e26b640 1 Processor -- start 2024-04-26T19:51:02.707 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.705+0000 7fa31e26b640 1 -- start start 2024-04-26T19:51:02.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa31e26b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180704c0 0x7fa3181f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.709 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa31e26b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3180708a0 0x7fa3180cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa31e26b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3180cca30 0x7fa31815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa31e26b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3180d1970 con 0x7fa3180704c0 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa31e26b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa31806fee0 con 0x7fa3180708a0 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa31e26b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa3180d1af0 con 0x7fa3180cca30 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa317fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180704c0 0x7fa3181f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa317fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180704c0 0x7fa3181f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46434/0 (socket says 172.21.15.79:46434) 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa317fff640 1 -- 172.21.15.79:0/1956114510 learned_addr learned my addr 172.21.15.79:0/1956114510 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa31c808640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3180cca30 0x7fa31815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.706+0000 7fa3177fe640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3180708a0 0x7fa3180cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa317fff640 1 -- 172.21.15.79:0/1956114510 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3180cca30 msgr2=0x7fa31815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa317fff640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3180cca30 0x7fa31815d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.710 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa317fff640 1 -- 172.21.15.79:0/1956114510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3180708a0 msgr2=0x7fa3180cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa317fff640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3180708a0 0x7fa3180cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa317fff640 1 -- 172.21.15.79:0/1956114510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3180ccf00 con 0x7fa3180704c0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa317fff640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180704c0 0x7fa3181f44d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c430 tx=0x7fa30800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa314ff9640 1 -- 172.21.15.79:0/1956114510 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308011790 con 0x7fa3180704c0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa314ff9640 1 -- 172.21.15.79:0/1956114510 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa308017070 con 0x7fa3180704c0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3180cd1e0 con 0x7fa3180704c0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa314ff9640 1 -- 172.21.15.79:0/1956114510 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308011bb0 con 0x7fa3180704c0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.707+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa318156b00 con 0x7fa3180704c0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.708+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3180506c0 con 0x7fa3180704c0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.708+0000 7fa314ff9640 1 -- 172.21.15.79:0/1956114510 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa30801c460 con 0x7fa3180704c0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.709+0000 7fa314ff9640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa30406c8f0 0x7fa30406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.709+0000 7fa314ff9640 1 -- 172.21.15.79:0/1956114510 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa30808bc00 con 0x7fa3180704c0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.709+0000 7fa3177fe640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa30406c8f0 0x7fa30406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:02.711 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.709+0000 7fa3177fe640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa30406c8f0 0x7fa30406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa3181f9360 tx=0x7fa30c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:02.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.713+0000 7fa314ff9640 1 -- 172.21.15.79:0/1956114510 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa308092080 con 0x7fa3180704c0 2024-04-26T19:51:02.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.924+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa318069bc0 con 0x7fa3180704c0 2024-04-26T19:51:02.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.925+0000 7fa314ff9640 1 -- 172.21.15.79:0/1956114510 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7fa3080596a0 con 0x7fa3180704c0 2024-04-26T19:51:02.926 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:02.926 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:52.517190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:51:02.926 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa30406c8f0 msgr2=0x7fa30406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa30406c8f0 0x7fa30406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa3181f9360 tx=0x7fa30c008040 comp rx=0 tx=0).stop 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180704c0 msgr2=0x7fa3181f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180704c0 0x7fa3181f44d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c430 tx=0x7fa30800c8f0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 shutdown_connections 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa30406c8f0 0x7fa30406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3180cca30 0x7fa31815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa3180708a0 0x7fa3180cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 --2- 172.21.15.79:0/1956114510 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3180704c0 0x7fa3181f44d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 >> 172.21.15.79:0/1956114510 conn(0x7fa318073820 msgr2=0x7fa31815e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:02.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 shutdown_connections 2024-04-26T19:51:02.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:02.927+0000 7fa31e26b640 1 -- 172.21.15.79:0/1956114510 wait complete. 2024-04-26T19:51:02.939 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:03.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.034+0000 7f3b07ca9640 1 Processor -- start 2024-04-26T19:51:03.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.035+0000 7f3b07ca9640 1 -- start start 2024-04-26T19:51:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.035+0000 7f3b07ca9640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3b00159890 0x7f3b00163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.035+0000 7f3b07ca9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b00159c70 0x7f3b0015fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.035+0000 7f3b07ca9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 0x7f3b000b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.035+0000 7f3b07ca9640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f3b00160280 con 0x7f3b00159890 2024-04-26T19:51:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.035+0000 7f3b07ca9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3b00160560 con 0x7f3b00159c70 2024-04-26T19:51:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.035+0000 7f3b07ca9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3b00160840 con 0x7f3b000b3b40 2024-04-26T19:51:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.035+0000 7f3b05a45640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3b00159890 0x7f3b00163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05a45640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3b00159890 0x7f3b00163a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46446/0 (socket says 172.21.15.79:46446) 2024-04-26T19:51:03.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05a45640 1 -- 172.21.15.79:0/496768843 learned_addr learned my addr 172.21.15.79:0/496768843 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b06246640 1 --2- 172.21.15.79:0/496768843 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 0x7f3b000b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05244640 1 --2- 172.21.15.79:0/496768843 >> v2:172.21.15.83:3300/0 conn(0x7f3b00159c70 0x7f3b0015fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05a45640 1 -- 172.21.15.79:0/496768843 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 msgr2=0x7f3b000b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05a45640 1 --2- 172.21.15.79:0/496768843 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 0x7f3b000b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05a45640 1 -- 172.21.15.79:0/496768843 >> v2:172.21.15.83:3300/0 conn(0x7f3b00159c70 msgr2=0x7f3b0015fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05a45640 1 --2- 172.21.15.79:0/496768843 >> v2:172.21.15.83:3300/0 conn(0x7f3b00159c70 0x7f3b0015fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05a45640 1 -- 172.21.15.79:0/496768843 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b00160e20 con 0x7f3b00159890 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05244640 1 --2- 172.21.15.79:0/496768843 >> v2:172.21.15.83:3300/0 conn(0x7f3b00159c70 0x7f3b0015fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.036+0000 7f3b05a45640 1 --2- 172.21.15.79:0/496768843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 0x7f3b00163a50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b00073420 tx=0x7f3afc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e0565147081809 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b04a43640 1 -- 172.21.15.79:0/496768843 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc037070 con 0x7f3b00159890 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b04a43640 1 -- 172.21.15.79:0/496768843 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3afc032440 con 0x7f3b00159890 2024-04-26T19:51:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b04a43640 1 -- 172.21.15.79:0/496768843 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3afc032810 con 0x7f3b00159890 2024-04-26T19:51:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b07ca9640 1 -- 172.21.15.79:0/496768843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 msgr2=0x7f3b00163a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/496768843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 0x7f3b00163a50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b00073420 tx=0x7f3afc02a1e0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b07ca9640 1 -- 172.21.15.79:0/496768843 shutdown_connections 2024-04-26T19:51:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/496768843 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 0x7f3b000b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/496768843 >> v2:172.21.15.83:3300/0 conn(0x7f3b00159c70 0x7f3b0015fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/496768843 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 0x7f3b00163a50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b07ca9640 1 -- 172.21.15.79:0/496768843 >> 172.21.15.79:0/496768843 conn(0x7f3b000b2720 msgr2=0x7f3b00070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b07ca9640 1 -- 172.21.15.79:0/496768843 shutdown_connections 2024-04-26T19:51:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.037+0000 7f3b07ca9640 1 -- 172.21.15.79:0/496768843 wait complete. 2024-04-26T19:51:03.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.038+0000 7f3b07ca9640 1 Processor -- start 2024-04-26T19:51:03.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.038+0000 7f3b07ca9640 1 -- start start 2024-04-26T19:51:03.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b07ca9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 0x7f3b001f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b07ca9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 0x7f3b0015b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b07ca9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b00159c70 0x7f3b000c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b07ca9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0006c350 con 0x7f3b00159890 2024-04-26T19:51:03.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b07ca9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0015ff60 con 0x7f3b00159c70 2024-04-26T19:51:03.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b07ca9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0006c4d0 con 0x7f3b000b3b40 2024-04-26T19:51:03.043 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b05244640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 0x7f3b0015b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b05244640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 0x7f3b0015b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46460/0 (socket says 172.21.15.79:46460) 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b05244640 1 -- 172.21.15.79:0/411405328 learned_addr learned my addr 172.21.15.79:0/411405328 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b05a45640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 0x7f3b001f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b06246640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b00159c70 0x7f3b000c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b05244640 1 -- 172.21.15.79:0/411405328 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 msgr2=0x7f3b001f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.039+0000 7f3b05244640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 0x7f3b001f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3b05244640 1 -- 172.21.15.79:0/411405328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b00159c70 msgr2=0x7f3b000c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3b05244640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b00159c70 0x7f3b000c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3b05244640 1 -- 172.21.15.79:0/411405328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0015c980 con 0x7f3b00159890 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3b06246640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b00159c70 0x7f3b000c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3b05244640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 0x7f3b0015b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3af000d7d0 tx=0x7f3af000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3af67fc640 1 -- 172.21.15.79:0/411405328 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0004030 con 0x7f3b00159890 2024-04-26T19:51:03.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3af67fc640 1 -- 172.21.15.79:0/411405328 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af00041d0 con 0x7f3b00159890 2024-04-26T19:51:03.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0015b610 con 0x7f3b00159890 2024-04-26T19:51:03.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3af67fc640 1 -- 172.21.15.79:0/411405328 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af00096a0 con 0x7f3b00159890 2024-04-26T19:51:03.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.040+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0015bea0 con 0x7f3b00159890 2024-04-26T19:51:03.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.041+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b000506c0 con 0x7f3b00159890 2024-04-26T19:51:03.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.041+0000 7f3af67fc640 1 -- 172.21.15.79:0/411405328 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3af0009840 con 0x7f3b00159890 2024-04-26T19:51:03.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.042+0000 7f3af67fc640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ae406c8f0 0x7f3ae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.042+0000 7f3b05a45640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ae406c8f0 0x7f3ae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.042+0000 7f3af67fc640 1 -- 172.21.15.79:0/411405328 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3af008c130 con 0x7f3b00159890 2024-04-26T19:51:03.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.042+0000 7f3b05a45640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ae406c8f0 0x7f3ae406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3afc034000 tx=0x7f3afc02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:03.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.046+0000 7f3af67fc640 1 -- 172.21.15.79:0/411405328 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3af005a220 con 0x7f3b00159890 2024-04-26T19:51:03.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.228+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b000afd90 con 0x7f3b00159890 2024-04-26T19:51:03.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.229+0000 7f3af67fc640 1 -- 172.21.15.79:0/411405328 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18466 (secure 0 0 0) 0x7f3af0059b20 con 0x7f3b00159890 2024-04-26T19:51:03.230 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:03.230 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":25,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:50:37.125661+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:03.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ae406c8f0 msgr2=0x7f3ae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ae406c8f0 0x7f3ae406edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3afc034000 tx=0x7f3afc02e000 comp rx=0 tx=0).stop 2024-04-26T19:51:03.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 msgr2=0x7f3b0015b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 0x7f3b0015b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3af000d7d0 tx=0x7f3af000dc90 comp rx=0 tx=0).stop 2024-04-26T19:51:03.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 shutdown_connections 2024-04-26T19:51:03.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ae406c8f0 0x7f3ae406edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b00159c70 0x7f3b000c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b00159890 0x7f3b0015b0d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 --2- 172.21.15.79:0/411405328 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b000b3b40 0x7f3b001f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 >> 172.21.15.79:0/411405328 conn(0x7f3b000b2720 msgr2=0x7f3b00070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:03.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 shutdown_connections 2024-04-26T19:51:03.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.231+0000 7f3b07ca9640 1 -- 172.21.15.79:0/411405328 wait complete. 2024-04-26T19:51:03.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:51:03.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.339+0000 7f055e086640 1 Processor -- start 2024-04-26T19:51:03.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.340+0000 7f055e086640 1 -- start start 2024-04-26T19:51:03.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.340+0000 7f055e086640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f055806fb20 0x7f05580d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.340+0000 7f055e086640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f055806ff00 0x7f05580d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.340+0000 7f055e086640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05581488c0 0x7f0558148d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.340+0000 7f055e086640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f05580d3e60 con 0x7f055806fb20 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.340+0000 7f055e086640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f05580b1290 con 0x7f055806ff00 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.340+0000 7f055e086640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f05580d4090 con 0x7f05581488c0 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.340+0000 7f05577fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f055806fb20 0x7f05580d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f05577fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f055806fb20 0x7f05580d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46476/0 (socket says 172.21.15.79:46476) 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f05577fe640 1 -- 172.21.15.79:0/111071420 learned_addr learned my addr 172.21.15.79:0/111071420 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f0556ffd640 1 --2- 172.21.15.79:0/111071420 >> v2:172.21.15.83:3300/0 conn(0x7f055806ff00 0x7f05580d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f0557fff640 1 --2- 172.21.15.79:0/111071420 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05581488c0 0x7f0558148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f05577fe640 1 -- 172.21.15.79:0/111071420 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05581488c0 msgr2=0x7f0558148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:03.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f05577fe640 1 --2- 172.21.15.79:0/111071420 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05581488c0 0x7f0558148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f05577fe640 1 -- 172.21.15.79:0/111071420 >> v2:172.21.15.83:3300/0 conn(0x7f055806ff00 msgr2=0x7f05580d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:03.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f05577fe640 1 --2- 172.21.15.79:0/111071420 >> v2:172.21.15.83:3300/0 conn(0x7f055806ff00 0x7f05580d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f05577fe640 1 -- 172.21.15.79:0/111071420 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05580d4370 con 0x7f055806fb20 2024-04-26T19:51:03.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f0556ffd640 1 --2- 172.21.15.79:0/111071420 >> v2:172.21.15.83:3300/0 conn(0x7f055806ff00 0x7f05580d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:03.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f0557fff640 1 --2- 172.21.15.79:0/111071420 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05581488c0 0x7f0558148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:03.345 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.341+0000 7f05577fe640 1 --2- 172.21.15.79:0/111071420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 0x7f05580d2cd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0548009870 tx=0x7f054802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da61f9b3c31cba87 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:03.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f05567fc640 1 -- 172.21.15.79:0/111071420 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548037070 con 0x7f055806fb20 2024-04-26T19:51:03.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f05567fc640 1 -- 172.21.15.79:0/111071420 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0548032440 con 0x7f055806fb20 2024-04-26T19:51:03.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f05567fc640 1 -- 172.21.15.79:0/111071420 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548032810 con 0x7f055806fb20 2024-04-26T19:51:03.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f055e086640 1 -- 172.21.15.79:0/111071420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 msgr2=0x7f05580d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f055e086640 1 --2- 172.21.15.79:0/111071420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 0x7f05580d2cd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0548009870 tx=0x7f054802a2a0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f055e086640 1 -- 172.21.15.79:0/111071420 shutdown_connections 2024-04-26T19:51:03.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f055e086640 1 --2- 172.21.15.79:0/111071420 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f05581488c0 0x7f0558148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f055e086640 1 --2- 172.21.15.79:0/111071420 >> v2:172.21.15.83:3300/0 conn(0x7f055806ff00 0x7f05580d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f055e086640 1 --2- 172.21.15.79:0/111071420 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 0x7f05580d2cd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f055e086640 1 -- 172.21.15.79:0/111071420 >> 172.21.15.79:0/111071420 conn(0x7f0558071c30 msgr2=0x7f055806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:03.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f055e086640 1 -- 172.21.15.79:0/111071420 shutdown_connections 2024-04-26T19:51:03.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.342+0000 7f055e086640 1 -- 172.21.15.79:0/111071420 wait complete. 2024-04-26T19:51:03.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.343+0000 7f055e086640 1 Processor -- start 2024-04-26T19:51:03.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.343+0000 7f055e086640 1 -- start start 2024-04-26T19:51:03.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f055e086640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 0x7f055815bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f055e086640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f055806ff00 0x7f055813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f055e086640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05581488c0 0x7f055813c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f055e086640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f05580d4c10 con 0x7f055806fb20 2024-04-26T19:51:03.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f055e086640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f05580d4930 con 0x7f05581488c0 2024-04-26T19:51:03.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f055e086640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0558149270 con 0x7f055806ff00 2024-04-26T19:51:03.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f05577fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 0x7f055815bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f05577fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 0x7f055815bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46484/0 (socket says 172.21.15.79:46484) 2024-04-26T19:51:03.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f05577fe640 1 -- 172.21.15.79:0/120429860 learned_addr learned my addr 172.21.15.79:0/120429860 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:03.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f0556ffd640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f055806ff00 0x7f055813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.344+0000 7f0557fff640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05581488c0 0x7f055813c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f05577fe640 1 -- 172.21.15.79:0/120429860 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f055806ff00 msgr2=0x7f055813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f05577fe640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f055806ff00 0x7f055813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f05577fe640 1 -- 172.21.15.79:0/120429860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05581488c0 msgr2=0x7f055813c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f05577fe640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05581488c0 0x7f055813c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f05577fe640 1 -- 172.21.15.79:0/120429860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0558161be0 con 0x7f055806fb20 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f05577fe640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 0x7f055815bef0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f054802a2e0 tx=0x7f05480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f053ffff640 1 -- 172.21.15.79:0/120429860 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548037040 con 0x7f055806fb20 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f053ffff640 1 -- 172.21.15.79:0/120429860 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0548042070 con 0x7f055806fb20 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f053ffff640 1 -- 172.21.15.79:0/120429860 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548030bd0 con 0x7f055806fb20 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055813c880 con 0x7f055806fb20 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.345+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055813ce40 con 0x7f055806fb20 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.346+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055813ebc0 con 0x7f055806fb20 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.346+0000 7f053ffff640 1 -- 172.21.15.79:0/120429860 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0548047460 con 0x7f055806fb20 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.347+0000 7f053ffff640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f054406c8f0 0x7f054406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.347+0000 7f053ffff640 1 -- 172.21.15.79:0/120429860 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f05480ac900 con 0x7f055806fb20 2024-04-26T19:51:03.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.347+0000 7f0556ffd640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f054406c8f0 0x7f054406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:03.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.347+0000 7f0556ffd640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f054406c8f0 0x7f054406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f05580b2bc0 tx=0x7f054c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:03.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.351+0000 7f053ffff640 1 -- 172.21.15.79:0/120429860 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f054807a430 con 0x7f055806fb20 2024-04-26T19:51:03.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.564+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05580b1f30 con 0x7f055806fb20 2024-04-26T19:51:03.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.564+0000 7f053ffff640 1 -- 172.21.15.79:0/120429860 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f054807e730 con 0x7f055806fb20 2024-04-26T19:51:03.565 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:03.565 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:52.517190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:51:03.565 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2024-04-26T19:51:03.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f054406c8f0 msgr2=0x7f054406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f054406c8f0 0x7f054406edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f05580b2bc0 tx=0x7f054c008040 comp rx=0 tx=0).stop 2024-04-26T19:51:03.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 msgr2=0x7f055815bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 0x7f055815bef0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f054802a2e0 tx=0x7f05480046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 shutdown_connections 2024-04-26T19:51:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f054406c8f0 0x7f054406edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f05581488c0 0x7f055813c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f055806ff00 0x7f055813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 --2- 172.21.15.79:0/120429860 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f055806fb20 0x7f055815bef0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.566+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 >> 172.21.15.79:0/120429860 conn(0x7f0558071c30 msgr2=0x7f055813ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.567+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 shutdown_connections 2024-04-26T19:51:03.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:03.567+0000 7f055e086640 1 -- 172.21.15.79:0/120429860 wait complete. 2024-04-26T19:51:03.981 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T19:50:30.451088+0000', 'modified': '2024-04-26T19:50:52.517190+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4302}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4302': {'gid': 4302, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.83:6835/3159271692', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.83:6833', 'nonce': 3159271692}, {'type': 'v1', 'addr': '172.21.15.83:6835', 'nonce': 3159271692}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4302, 'qdb_cluster': [4302]} 2024-04-26T19:51:03.982 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T19:51:03.982 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-26T19:51:04.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.081+0000 7f854b311640 1 Processor -- start 2024-04-26T19:51:04.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.081+0000 7f854b311640 1 -- start start 2024-04-26T19:51:04.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f854b311640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8544160390 0x7f85440b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f854b311640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f85441637e0 0x7f85440b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f854b311640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8544144ca0 con 0x7f8544160770 2024-04-26T19:51:04.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f854b311640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8544163e00 con 0x7f8544160390 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f854b311640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8544144f30 con 0x7f85441637e0 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f85488ac640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f8544160770 0x7f85441630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53148/0 (socket says 172.21.15.79:53148) 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f85488ac640 1 -- 172.21.15.79:0/2313126643 learned_addr learned my addr 172.21.15.79:0/2313126643 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 551982804 0 0) 0x7f8544144ca0 con 0x7f8544160770 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f85490ad640 1 --2- 172.21.15.79:0/2313126643 >> v2:172.21.15.83:3300/0 conn(0x7f8544160390 0x7f85440b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f85498ae640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f85441637e0 0x7f85440b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.082+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8530003520 con 0x7f8544160770 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1892378672 0 0) 0x7f8530003520 con 0x7f8544160770 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f85441637e0 msgr2=0x7f85440b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.086 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f85441637e0 0x7f85440b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 >> v2:172.21.15.83:3300/0 conn(0x7f8544160390 msgr2=0x7f85440b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 --2- 172.21.15.79:0/2313126643 >> v2:172.21.15.83:3300/0 conn(0x7f8544160390 0x7f85440b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8544145210 con 0x7f8544160770 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f8534001670 con 0x7f8544160770 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 >> v1:172.21.15.79:6789/0 conn(0x7f8544160770 legacy=0x7f85441630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85300039b0 0x7f8530003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f85441637e0 0x7f8530004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f85300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8544163e00 con 0x7f8530004d40 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8544144f30 con 0x7f85300039b0 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f853000bb20 con 0x7f85441637e0 2024-04-26T19:51:04.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f85498ae640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f85300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f85488ac640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f85441637e0 0x7f8530004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f85490ad640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85300039b0 0x7f8530003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f85498ae640 1 -- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f85441637e0 msgr2=0x7f8530004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f85498ae640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f85441637e0 0x7f8530004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f85498ae640 1 -- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85300039b0 msgr2=0x7f8530003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f85498ae640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85300039b0 0x7f8530003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.083+0000 7f85498ae640 1 -- 172.21.15.79:0/2313126643 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8530003520 con 0x7f8530004d40 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.084+0000 7f85498ae640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f85300051c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f853c00ba30 tx=0x7f8530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8197d67829597b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.084+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c030070 con 0x7f8530004d40 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.084+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853c02ade0 con 0x7f8530004d40 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.084+0000 7f853bfff640 1 -- 172.21.15.79:0/2313126643 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c00b600 con 0x7f8530004d40 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.084+0000 7f854b311640 1 -- 172.21.15.79:0/2313126643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 msgr2=0x7f85300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.084+0000 7f854b311640 1 --2- 172.21.15.79:0/2313126643 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f85300051c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f853c00ba30 tx=0x7f8530005830 comp rx=0 tx=0).stop 2024-04-26T19:51:04.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.084+0000 7f85498ae640 1 -- 172.21.15.79:0/2313126643 reap_dead start 2024-04-26T19:51:04.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.084+0000 7f854b311640 1 -- 172.21.15.79:0/2313126643 shutdown_connections 2024-04-26T19:51:04.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.084+0000 7f854b311640 1 -- 172.21.15.79:0/2313126643 >> 172.21.15.79:0/2313126643 conn(0x7f85440b2720 msgr2=0x7f8544070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:04.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.085+0000 7f854b311640 1 -- 172.21.15.79:0/2313126643 shutdown_connections 2024-04-26T19:51:04.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.085+0000 7f854b311640 1 -- 172.21.15.79:0/2313126643 wait complete. 2024-04-26T19:51:04.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f854b311640 1 Processor -- start 2024-04-26T19:51:04.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f854b311640 1 -- start start 2024-04-26T19:51:04.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f854b311640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f8544133bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f854b311640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8544148850 0x7f85440aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f854b311640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85440b0ac0 0x7f85440b0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f854b311640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8544145b90 con 0x7f8530004d40 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f854b311640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8544145a10 con 0x7f85440b0ac0 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f854b311640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f85440b4050 con 0x7f8544148850 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f85490ad640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f8544133bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f85490ad640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f8544133bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46500/0 (socket says 172.21.15.79:46500) 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.086+0000 7f85490ad640 1 -- 172.21.15.79:0/1826907207 learned_addr learned my addr 172.21.15.79:0/1826907207 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85488ac640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8544148850 0x7f85440aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85498ae640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85440b0ac0 0x7f85440b0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85490ad640 1 -- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8544148850 msgr2=0x7f85440aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85490ad640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8544148850 0x7f85440aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85490ad640 1 -- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85440b0ac0 msgr2=0x7f85440b0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85490ad640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85440b0ac0 0x7f85440b0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85490ad640 1 -- 172.21.15.79:0/1826907207 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85440b1790 con 0x7f8530004d40 2024-04-26T19:51:04.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85498ae640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85440b0ac0 0x7f85440b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:04.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85488ac640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8544148850 0x7f85440aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:04.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f85490ad640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f8544133bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f85400046a0 tx=0x7f8540007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f8539ffb640 1 -- 172.21.15.79:0/1826907207 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540015420 con 0x7f8530004d40 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f8539ffb640 1 -- 172.21.15.79:0/1826907207 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854001a070 con 0x7f8530004d40 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f854b311640 1 -- 172.21.15.79:0/1826907207 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85440b1a30 con 0x7f8530004d40 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f8539ffb640 1 -- 172.21.15.79:0/1826907207 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85400157e0 con 0x7f8530004d40 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.087+0000 7f854b311640 1 -- 172.21.15.79:0/1826907207 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854415a0d0 con 0x7f8530004d40 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.088+0000 7f8539ffb640 1 -- 172.21.15.79:0/1826907207 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8540015980 con 0x7f8530004d40 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.089+0000 7f8539ffb640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f852806c8f0 0x7f852806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.089+0000 7f8539ffb640 1 -- 172.21.15.79:0/1826907207 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f854008c900 con 0x7f8530004d40 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.089+0000 7f85488ac640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f852806c8f0 0x7f852806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.089+0000 7f85488ac640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f852806c8f0 0x7f852806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f854415c9b0 tx=0x7f8534009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.090+0000 7f854e021740 1 Processor -- start 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.090+0000 7f854e021740 1 -- start start 2024-04-26T19:51:04.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f854e021740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8b1440 0x55b0fb8b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f854e021740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8cece0 0x55b0fb8cf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f854e021740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb887120 0x55b0fb8cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f854e021740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b0fb7eb740 con 0x55b0fb8b1440 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f854e021740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b0fb7873a0 con 0x55b0fb887120 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f854e021740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x55b0fb7e7700 con 0x55b0fb8cece0 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f85490ad640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8b1440 0x55b0fb8b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f85490ad640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8b1440 0x55b0fb8b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46506/0 (socket says 172.21.15.79:46506) 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f85490ad640 1 -- 172.21.15.79:0/1064506647 learned_addr learned my addr 172.21.15.79:0/1064506647 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f85490ad640 1 -- 172.21.15.79:0/1064506647 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8cece0 msgr2=0x55b0fb8cf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f85490ad640 1 --2- 172.21.15.79:0/1064506647 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8cece0 0x55b0fb8cf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f85490ad640 1 -- 172.21.15.79:0/1064506647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb887120 msgr2=0x55b0fb8cd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f85490ad640 1 --2- 172.21.15.79:0/1064506647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb887120 0x55b0fb8cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f85490ad640 1 -- 172.21.15.79:0/1064506647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fb7a3750 con 0x55b0fb8b1440 2024-04-26T19:51:04.102 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f85490ad640 1 --2- 172.21.15.79:0/1064506647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8b1440 0x55b0fb8b1820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8540009730 tx=0x7f854005aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f4bad6525ad54a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f852effd640 1 -- 172.21.15.79:0/1064506647 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540062ed0 con 0x55b0fb8b1440 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f852effd640 1 -- 172.21.15.79:0/1064506647 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85400585f0 con 0x55b0fb8b1440 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.091+0000 7f852effd640 1 -- 172.21.15.79:0/1064506647 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540056960 con 0x55b0fb8b1440 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.092+0000 7f854e021740 1 -- 172.21.15.79:0/1064506647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8b1440 msgr2=0x55b0fb8b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.092+0000 7f854e021740 1 --2- 172.21.15.79:0/1064506647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8b1440 0x55b0fb8b1820 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8540009730 tx=0x7f854005aab0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.092+0000 7f854e021740 1 -- 172.21.15.79:0/1064506647 shutdown_connections 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.092+0000 7f854e021740 1 --2- 172.21.15.79:0/1064506647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb887120 0x55b0fb8cd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.092+0000 7f854e021740 1 --2- 172.21.15.79:0/1064506647 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8cece0 0x55b0fb8cf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.092+0000 7f854e021740 1 --2- 172.21.15.79:0/1064506647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8b1440 0x55b0fb8b1820 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.092+0000 7f854e021740 1 -- 172.21.15.79:0/1064506647 >> 172.21.15.79:0/1064506647 conn(0x55b0fb883a50 msgr2=0x55b0fb8cc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.092+0000 7f854e021740 1 -- 172.21.15.79:0/1064506647 shutdown_connections 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.092+0000 7f854e021740 1 -- 172.21.15.79:0/1064506647 wait complete. 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.093+0000 7f854e021740 1 -- start start 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.093+0000 7f854e021740 1 Processor -- start 2024-04-26T19:51:04.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.093+0000 7f854e021740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8b1440 0x55b0fb8dc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.093+0000 7f854e021740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.093+0000 7f854e021740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8c7350 0x55b0fb8c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.093+0000 7f854e021740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b0fb887ec0 con 0x55b0fb8cece0 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.093+0000 7f854e021740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b0fb772fe0 con 0x55b0fb8c7350 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f854e021740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x55b0fb881980 con 0x55b0fb8b1440 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f854e021740 10 client.0 _ll_register_callbacks cb 0x55b0fb789150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85488ac640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85488ac640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46514/0 (socket says 172.21.15.79:46514) 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85488ac640 1 -- 172.21.15.79:0/3165345501 learned_addr learned my addr 172.21.15.79:0/3165345501 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85490ad640 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8b1440 0x55b0fb8dc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85498ae640 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8c7350 0x55b0fb8c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85488ac640 1 -- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8b1440 msgr2=0x55b0fb8dc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85488ac640 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8b1440 0x55b0fb8dc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85488ac640 1 -- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8c7350 msgr2=0x55b0fb8c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85488ac640 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8c7350 0x55b0fb8c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85488ac640 1 -- 172.21.15.79:0/3165345501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fb4fd2d0 con 0x55b0fb8cece0 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85498ae640 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8c7350 0x55b0fb8c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f85488ac640 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8a4a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8534015760 tx=0x7f85340078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6099705672a57c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f8523fff640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f8523fff640 1 -- 172.21.15.79:0/3165345501 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534026060 con 0x55b0fb8cece0 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f8523fff640 1 -- 172.21.15.79:0/3165345501 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f85340147a0 con 0x55b0fb8cece0 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f854e021740 1 -- 172.21.15.79:0/3165345501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85400585f0 con 0x55b0fb8cece0 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f8523fff640 1 -- 172.21.15.79:0/3165345501 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f853402c050 con 0x55b0fb8cece0 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f854e021740 10 client.4860 fetch_fsmap learned FSMap version 9 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f854e021740 1 -- 172.21.15.79:0/3165345501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b0fb5c76b0 con 0x55b0fb8cece0 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.094+0000 7f8523fff640 1 -- 172.21.15.79:0/3165345501 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85400585f0 con 0x55b0fb8cece0 2024-04-26T19:51:04.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f8523fff640 1 -- 172.21.15.79:0/3165345501 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f853402f020 con 0x55b0fb8cece0 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f854e021740 10 client.4860 fetch_fsmap finished waiting for FSMap version 9 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f854e021740 10 client.4860 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f854e021740 20 client.4860 populate_metadata read hostname 'smithi079' 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f854e021740 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55b0fb8dfcf0 0x55b0fb8e00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f854e021740 4 client.4860 mds_command: new command op to 4302 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f854e021740 1 -- 172.21.15.79:0/3165345501 --> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0fb7f9350 con 0x55b0fb8dfcf0 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f85490ad640 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55b0fb8dfcf0 0x55b0fb8e00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f8523fff640 10 client.4860 ms_handle_connect on v2:172.21.15.83:6833/3159271692 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.095+0000 7f85490ad640 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55b0fb8dfcf0 0x55b0fb8e00d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=375759c93232ccd0 server_cookie=693f7f586a86cd49 in_seq=0 out_seq=0 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f8523fff640 1 -- 172.21.15.79:0/3165345501 <== mds.0 v2:172.21.15.83:6833/3159271692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b0fb7f9350 con 0x55b0fb8dfcf0 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f8523fff640 10 client.4860 handle_command_reply: tid=0 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 client.4860 shutdown 2024-04-26T19:51:04.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 -- 172.21.15.79:0/3165345501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 msgr2=0x55b0fb8a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8a4a90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8534015760 tx=0x7f85340078f0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 -- 172.21.15.79:0/3165345501 shutdown_connections 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55b0fb8dfcf0 0x55b0fb8e00d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8c7350 0x55b0fb8c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8a4a90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 --2- 172.21.15.79:0/3165345501 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8b1440 0x55b0fb8dc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f8523fff640 0 client.4860 ms_handle_reset on v2:172.21.15.83:6833/3159271692 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 -- 172.21.15.79:0/3165345501 >> 172.21.15.79:0/3165345501 conn(0x55b0fb887120 msgr2=0x55b0fb8d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 -- 172.21.15.79:0/3165345501 shutdown_connections 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 1 -- 172.21.15.79:0/3165345501 wait complete. 2024-04-26T19:51:04.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.096+0000 7f854e021740 20 client.4860 trim_cache size 0 max 16384 2024-04-26T19:51:04.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.114+0000 7f854e021740 1 Processor -- start 2024-04-26T19:51:04.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.114+0000 7f854e021740 1 -- start start 2024-04-26T19:51:04.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f854e021740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 0x55b0fb8dc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f854e021740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8c7350 0x55b0fb8dcb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.117 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f854e021740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8cece0 0x55b0fb887520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f854e021740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8544202440 con 0x55b0fb8c7350 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f854e021740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f85442017d0 con 0x55b0fb8b1440 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f854e021740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8544201e20 con 0x55b0fb8cece0 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85488ac640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8c7350 0x55b0fb8dcb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85488ac640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8c7350 0x55b0fb8dcb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46526/0 (socket says 172.21.15.79:46526) 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85488ac640 1 -- 172.21.15.79:0/348685018 learned_addr learned my addr 172.21.15.79:0/348685018 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85490ad640 1 --2- 172.21.15.79:0/348685018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 0x55b0fb8dc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85498ae640 1 --2- 172.21.15.79:0/348685018 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8cece0 0x55b0fb887520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85488ac640 1 -- 172.21.15.79:0/348685018 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8cece0 msgr2=0x55b0fb887520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85488ac640 1 --2- 172.21.15.79:0/348685018 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8cece0 0x55b0fb887520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85488ac640 1 -- 172.21.15.79:0/348685018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 msgr2=0x55b0fb8dc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85488ac640 1 --2- 172.21.15.79:0/348685018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 0x55b0fb8dc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85488ac640 1 -- 172.21.15.79:0/348685018 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fb8c80d0 con 0x55b0fb8c7350 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f85488ac640 1 --2- 172.21.15.79:0/348685018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8c7350 0x55b0fb8dcb80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f853402a040 tx=0x7f8534017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300da994e1bbff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f852e7fc640 1 -- 172.21.15.79:0/348685018 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534027450 con 0x55b0fb8c7350 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f852e7fc640 1 -- 172.21.15.79:0/348685018 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853401c050 con 0x55b0fb8c7350 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.115+0000 7f852e7fc640 1 -- 172.21.15.79:0/348685018 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340277f0 con 0x55b0fb8c7350 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.116+0000 7f854e021740 1 -- 172.21.15.79:0/348685018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8c7350 msgr2=0x55b0fb8dcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.116+0000 7f854e021740 1 --2- 172.21.15.79:0/348685018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8c7350 0x55b0fb8dcb80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f853402a040 tx=0x7f8534017000 comp rx=0 tx=0).stop 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.116+0000 7f854e021740 1 -- 172.21.15.79:0/348685018 shutdown_connections 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.116+0000 7f854e021740 1 --2- 172.21.15.79:0/348685018 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8cece0 0x55b0fb887520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.116+0000 7f854e021740 1 --2- 172.21.15.79:0/348685018 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8c7350 0x55b0fb8dcb80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.116+0000 7f854e021740 1 --2- 172.21.15.79:0/348685018 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 0x55b0fb8dc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.116+0000 7f854e021740 1 -- 172.21.15.79:0/348685018 >> 172.21.15.79:0/348685018 conn(0x55b0fb887120 msgr2=0x55b0fb884ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.116+0000 7f854e021740 1 -- 172.21.15.79:0/348685018 shutdown_connections 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.116+0000 7f854e021740 1 -- 172.21.15.79:0/348685018 wait complete. 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.117+0000 7f854e021740 1 -- start start 2024-04-26T19:51:04.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.117+0000 7f854e021740 1 Processor -- start 2024-04-26T19:51:04.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.117+0000 7f854e021740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 0x55b0fb8cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.117+0000 7f854e021740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8c7350 0x55b0fb8b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.117+0000 7f854e021740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.117+0000 7f854e021740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55b0fb8c3f50 con 0x55b0fb8cece0 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.117+0000 7f854e021740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55b0fb8c6e80 con 0x55b0fb8b1440 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.117+0000 7f854e021740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x55b0fb8c3420 con 0x55b0fb8c7350 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f854e021740 10 client.0 _ll_register_callbacks cb 0x7f8534018040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85498ae640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85498ae640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8c40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46536/0 (socket says 172.21.15.79:46536) 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85498ae640 1 -- 172.21.15.79:0/1626637167 learned_addr learned my addr 172.21.15.79:0/1626637167 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85490ad640 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 0x55b0fb8cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85488ac640 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8c7350 0x55b0fb8b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85498ae640 1 -- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8c7350 msgr2=0x55b0fb8b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85498ae640 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8c7350 0x55b0fb8b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85498ae640 1 -- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 msgr2=0x55b0fb8cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85498ae640 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 0x55b0fb8cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85498ae640 1 -- 172.21.15.79:0/1626637167 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fb8cde40 con 0x55b0fb8cece0 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f85498ae640 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8c40d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f853c02a8d0 tx=0x7f853c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938fec694d96b5e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f8523fff640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f8523fff640 1 -- 172.21.15.79:0/1626637167 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c004020 con 0x55b0fb8cece0 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f8523fff640 1 -- 172.21.15.79:0/1626637167 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f853c0041c0 con 0x55b0fb8cece0 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f854e021740 1 -- 172.21.15.79:0/1626637167 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f853401c050 con 0x55b0fb8cece0 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f8523fff640 1 -- 172.21.15.79:0/1626637167 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f853c04c050 con 0x55b0fb8cece0 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f854e021740 10 client.4866 fetch_fsmap learned FSMap version 9 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.118+0000 7f854e021740 1 -- 172.21.15.79:0/1626637167 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55b0fb8cd440 con 0x55b0fb8cece0 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f8523fff640 1 -- 172.21.15.79:0/1626637167 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853401c050 con 0x55b0fb8cece0 2024-04-26T19:51:04.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f8523fff640 1 -- 172.21.15.79:0/1626637167 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 9) ==== 3502+0+0 (secure 0 0 0) 0x7f853c04f020 con 0x55b0fb8cece0 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f854e021740 10 client.4866 fetch_fsmap finished waiting for FSMap version 9 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f854e021740 10 client.4866 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f854e021740 20 client.4866 populate_metadata read hostname 'smithi079' 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f854e021740 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55b0fb886070 0x55b0fb886450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f854e021740 4 client.4866 mds_command: new command op to 4302 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f854e021740 1 -- 172.21.15.79:0/1626637167 --> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] -- command(tid 0: {"prefix": "status"}) -- 0x55b0fb7f31c0 con 0x55b0fb886070 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f85490ad640 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55b0fb886070 0x55b0fb886450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f85490ad640 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55b0fb886070 0x55b0fb886450 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884dc194308b0823 server_cookie=d9f7be88c8209eac in_seq=0 out_seq=0 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f8523fff640 10 client.4866 ms_handle_connect on v2:172.21.15.83:6833/3159271692 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f8523fff640 1 -- 172.21.15.79:0/1626637167 <== mds.0 v2:172.21.15.83:6833/3159271692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55b0fb7f31c0 con 0x55b0fb886070 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f8523fff640 10 client.4866 handle_command_reply: tid=0 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f854e021740 1 client.4866 shutdown 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f854e021740 1 -- 172.21.15.79:0/1626637167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 msgr2=0x55b0fb8c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.119+0000 7f854e021740 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8c40d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f853c02a8d0 tx=0x7f853c004730 comp rx=0 tx=0).stop 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854e021740 1 -- 172.21.15.79:0/1626637167 shutdown_connections 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854e021740 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:6833/3159271692,v1:172.21.15.83:6835/3159271692] conn(0x55b0fb886070 0x55b0fb886450 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854e021740 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55b0fb8cece0 0x55b0fb8c40d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854e021740 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55b0fb8c7350 0x55b0fb8b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f8523fff640 0 client.4866 ms_handle_reset on v2:172.21.15.83:6833/3159271692 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854e021740 1 --2- 172.21.15.79:0/1626637167 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55b0fb8b1440 0x55b0fb8cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854e021740 1 -- 172.21.15.79:0/1626637167 >> 172.21.15.79:0/1626637167 conn(0x55b0fb887120 msgr2=0x55b0fb887500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854e021740 1 -- 172.21.15.79:0/1626637167 shutdown_connections 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854e021740 1 -- 172.21.15.79:0/1626637167 wait complete. 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854e021740 20 client.4866 trim_cache size 0 max 16384 2024-04-26T19:51:04.127 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "b372f41f-f07b-4764-9bef-bcff62f9a5d2", 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 4302, 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 33.014544557999997, 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 9, 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 25, 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 25, 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 40.038880962 2024-04-26T19:51:04.128 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:51:04.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 -- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f852806c8f0 msgr2=0x7f852806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f852806c8f0 0x7f852806edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f854415c9b0 tx=0x7f8534009040 comp rx=0 tx=0).stop 2024-04-26T19:51:04.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 -- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 msgr2=0x7f8544133bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f8544133bb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f85400046a0 tx=0x7f8540007700 comp rx=0 tx=0).stop 2024-04-26T19:51:04.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 -- 172.21.15.79:0/1826907207 shutdown_connections 2024-04-26T19:51:04.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f852806c8f0 0x7f852806edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f85440b0ac0 0x7f85440b0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8544148850 0x7f85440aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 --2- 172.21.15.79:0/1826907207 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8530004d40 0x7f8544133bb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 -- 172.21.15.79:0/1826907207 >> 172.21.15.79:0/1826907207 conn(0x7f85440b2720 msgr2=0x7f85441412d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:04.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 -- 172.21.15.79:0/1826907207 shutdown_connections 2024-04-26T19:51:04.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.120+0000 7f854b311640 1 -- 172.21.15.79:0/1826907207 wait complete. 2024-04-26T19:51:04.132 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T19:51:04.141 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T19:51:04.141 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T19:51:04.141 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T19:51:04.142 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T19:51:04.157 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T19:51:04.157 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-26T19:51:04.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.157 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-04-26T19:51:04.172 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-26T19:51:04.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.173 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-26T19:51:04.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.292 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2024-04-26T19:51:04.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.347 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-04-26T19:51:04.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.362 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2024-04-26T19:51:04.428 INFO:teuthology.orchestra.run.smithi079.stderr:Cannot find device "ceph-brx" 2024-04-26T19:51:04.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:51:04.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.429 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-26T19:51:04.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.448 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-26T19:51:04.520 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-04-26T19:51:04.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:51:04.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.522 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-26T19:51:04.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.580 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-26T19:51:04.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:51:04.648 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-04-26T19:51:04.649 INFO:tasks.kclient:client.0 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v1'} 2024-04-26T19:51:04.649 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T19:51:04.649 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T19:51:04.649 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T19:51:04.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.746+0000 7f81c2f51640 1 Processor -- start 2024-04-26T19:51:04.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.746+0000 7f81c2f51640 1 -- start start 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.747+0000 7f81c2f51640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f81bc0cd9c0 0x7f81bc0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.747+0000 7f81c2f51640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f81bc0735a0 0x7f81bc0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.747+0000 7f81c2f51640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81bc0ce1a0 con 0x7f81bc156670 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.747+0000 7f81c2f51640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f81bc073110 con 0x7f81bc0735a0 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.747+0000 7f81c2f51640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f81bc0cdec0 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.747+0000 7f81c0ced640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f81bc156670 0x7f81bc15d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53162/0 (socket says 172.21.15.79:53162) 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.747+0000 7f81c0ced640 1 -- 172.21.15.79:0/2726283036 learned_addr learned my addr 172.21.15.79:0/2726283036 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.747+0000 7f81c14ee640 1 --2- 172.21.15.79:0/2726283036 >> v2:172.21.15.79:3300/0 conn(0x7f81bc0735a0 0x7f81bc0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.747+0000 7f81bbfff640 1 --2- 172.21.15.79:0/2726283036 >> v2:172.21.15.83:3300/0 conn(0x7f81bc0cd9c0 0x7f81bc0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81bb7fe640 1 -- 172.21.15.79:0/2726283036 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 12032008 0 0) 0x7f81bc0ce1a0 con 0x7f81bc156670 2024-04-26T19:51:04.750 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81bb7fe640 1 -- 172.21.15.79:0/2726283036 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f81a4003520 con 0x7f81bc156670 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81c14ee640 1 -- 172.21.15.79:0/2726283036 >> v2:172.21.15.83:3300/0 conn(0x7f81bc0cd9c0 msgr2=0x7f81bc0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81c14ee640 1 --2- 172.21.15.79:0/2726283036 >> v2:172.21.15.83:3300/0 conn(0x7f81bc0cd9c0 0x7f81bc0b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81c14ee640 1 -- 172.21.15.79:0/2726283036 >> v1:172.21.15.79:6789/0 conn(0x7f81bc156670 legacy=0x7f81bc15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81c14ee640 1 -- 172.21.15.79:0/2726283036 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc072ac0 con 0x7f81bc0735a0 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81bbfff640 1 --2- 172.21.15.79:0/2726283036 >> v2:172.21.15.83:3300/0 conn(0x7f81bc0cd9c0 0x7f81bc0b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81c14ee640 1 --2- 172.21.15.79:0/2726283036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0735a0 0x7f81bc0b4320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f81bc074170 tx=0x7f81b402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2c5bc2d7c9fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81bb7fe640 1 -- 172.21.15.79:0/2726283036 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4007740 con 0x7f81bc0735a0 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81bb7fe640 1 -- 172.21.15.79:0/2726283036 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b40078e0 con 0x7f81bc0735a0 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.748+0000 7f81bb7fe640 1 -- 172.21.15.79:0/2726283036 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4007cb0 con 0x7f81bc0735a0 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.749+0000 7f81c2f51640 1 -- 172.21.15.79:0/2726283036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0735a0 msgr2=0x7f81bc0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.749+0000 7f81c2f51640 1 --2- 172.21.15.79:0/2726283036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0735a0 0x7f81bc0b4320 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f81bc074170 tx=0x7f81b402a680 comp rx=0 tx=0).stop 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.749+0000 7f81c2f51640 1 -- 172.21.15.79:0/2726283036 shutdown_connections 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.749+0000 7f81c2f51640 1 --2- 172.21.15.79:0/2726283036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0735a0 0x7f81bc0b4320 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.751 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.749+0000 7f81c2f51640 1 --2- 172.21.15.79:0/2726283036 >> v2:172.21.15.83:3300/0 conn(0x7f81bc0cd9c0 0x7f81bc0b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.749+0000 7f81c2f51640 1 -- 172.21.15.79:0/2726283036 >> 172.21.15.79:0/2726283036 conn(0x7f81bc0b39d0 msgr2=0x7f81bc06feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:04.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.749+0000 7f81c2f51640 1 -- 172.21.15.79:0/2726283036 shutdown_connections 2024-04-26T19:51:04.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.749+0000 7f81c2f51640 1 -- 172.21.15.79:0/2726283036 wait complete. 2024-04-26T19:51:04.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.750+0000 7f81c2f51640 1 Processor -- start 2024-04-26T19:51:04.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.750+0000 7f81c2f51640 1 -- start start 2024-04-26T19:51:04.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81c2f51640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f81bc0735a0 0x7f81bc1f2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81c2f51640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0cd9c0 0x7f81bc15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81c2f51640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81bc156670 0x7f81bc151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81c2f51640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc156bd0 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81c2f51640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f81bc156a50 con 0x7f81bc156670 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81c2f51640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f81bc0cf650 con 0x7f81bc0735a0 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81bbfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0cd9c0 0x7f81bc15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81bbfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0cd9c0 0x7f81bc15daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46546/0 (socket says 172.21.15.79:46546) 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81bbfff640 1 -- 172.21.15.79:0/548854191 learned_addr learned my addr 172.21.15.79:0/548854191 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81c0ced640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f81bc0735a0 0x7f81bc1f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.751+0000 7f81c14ee640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81bc156670 0x7f81bc151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81bbfff640 1 -- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f81bc0735a0 msgr2=0x7f81bc1f2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81bbfff640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f81bc0735a0 0x7f81bc1f2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81bbfff640 1 -- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81bc156670 msgr2=0x7f81bc151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81bbfff640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81bc156670 0x7f81bc151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81bbfff640 1 -- 172.21.15.79:0/548854191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81bc157530 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81c0ced640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f81bc0735a0 0x7f81bc1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81c14ee640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81bc156670 0x7f81bc151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81bbfff640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0cd9c0 0x7f81bc15daf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e840 tx=0x7f81b000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81b97fa640 1 -- 172.21.15.79:0/548854191 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b000f040 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81b97fa640 1 -- 172.21.15.79:0/548854191 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b0004030 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81b97fa640 1 -- 172.21.15.79:0/548854191 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0004400 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81bc15e540 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.752+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81bc1577f0 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.753+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81bc0506c0 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.753+0000 7f81b97fa640 1 -- 172.21.15.79:0/548854191 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f81b000c6d0 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.754+0000 7f81b97fa640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f81a806c8f0 0x7f81a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.754+0000 7f81b97fa640 1 -- 172.21.15.79:0/548854191 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f81b008bb80 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.754+0000 7f81c0ced640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f81a806c8f0 0x7f81a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:04.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.754+0000 7f81c0ced640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f81a806c8f0 0x7f81a806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f81ac007fd0 tx=0x7f81ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:04.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.758+0000 7f81b97fa640 1 -- 172.21.15.79:0/548854191 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f81b0017020 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.969+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f81bc160220 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.969+0000 7f81b97fa640 1 -- 172.21.15.79:0/548854191 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f81b0059c70 con 0x7f81bc0cd9c0 2024-04-26T19:51:04.970 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T19:51:04.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f81a806c8f0 msgr2=0x7f81a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f81a806c8f0 0x7f81a806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f81ac007fd0 tx=0x7f81ac009040 comp rx=0 tx=0).stop 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0cd9c0 msgr2=0x7f81bc15daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0cd9c0 0x7f81bc15daf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f81b000e840 tx=0x7f81b000ed00 comp rx=0 tx=0).stop 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 shutdown_connections 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f81a806c8f0 0x7f81a806edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f81bc156670 0x7f81bc151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f81bc0cd9c0 0x7f81bc15daf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 --2- 172.21.15.79:0/548854191 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f81bc0735a0 0x7f81bc1f2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 >> 172.21.15.79:0/548854191 conn(0x7f81bc0b39d0 msgr2=0x7f81bc151210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 shutdown_connections 2024-04-26T19:51:04.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:04.971+0000 7f81c2f51640 1 -- 172.21.15.79:0/548854191 wait complete. 2024-04-26T19:51:04.983 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T19:51:04.983 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T19:51:04.983 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2024-04-26T19:51:04.983 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T19:51:04.983 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T19:51:04.983 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T19:51:04.984 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T19:51:04.984 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T19:51:04.984 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T19:51:04.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:04.984 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f0 2024-04-26T19:51:04.999 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f1 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42684sec preferred_lft 42684sec 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::5909:6a14:c050:be1a/64 scope link noprefixroute 2024-04-26T19:51:05.000 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:05.000 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T19:51:05.001 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:05.001 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:05.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add name ceph-brx type bridge 2024-04-26T19:51:05.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr flush dev ceph-brx 2024-04-26T19:51:05.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set ceph-brx up 2024-04-26T19:51:05.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T19:51:05.001 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:05.087 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:05.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:05.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:05.133 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T19:51:05.154 INFO:teuthology.orchestra.run.smithi079.stdout:1 2024-04-26T19:51:05.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:05.156 DEBUG:teuthology.orchestra.run.smithi079:> route 2024-04-26T19:51:05.212 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2024-04-26T19:51:05.212 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T19:51:05.212 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T19:51:05.212 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T19:51:05.212 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T19:51:05.212 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:05.212 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:05.212 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T19:51:05.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T19:51:05.213 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T19:51:05.213 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:05.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:05.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:05.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:05.336 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-04-26T19:51:05.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:05.351 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2024-04-26T19:51:05.406 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:05.406 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:05.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T19:51:05.406 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T19:51:05.406 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:05.494 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:05.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:05.521 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T19:51:05.522 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:05.522 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:05.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T19:51:05.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T19:51:05.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T19:51:05.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T19:51:05.522 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T19:51:05.522 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:05.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:05.624 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:05.630 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:05.630 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:05.630 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2024-04-26T19:51:05.631 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T19:51:05.631 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:05.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:05.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:05.704 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:05.705 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T19:51:05.719 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T19:51:05.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:05.720 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T19:51:05.775 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T19:51:05.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:05.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-04-26T19:51:05.856 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-04-26T19:51:05.857 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-04-26T19:51:05.857 INFO:teuthology.orchestra.run.smithi079.stdout:invalid new device string format 2024-04-26T19:51:05.955 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-04-26T19:51:05.955 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-04-26T19:51:05.955 INFO:teuthology.orchestra.run.smithi079.stdout:invalid new device string format 2024-04-26T19:51:05.955 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:05.955 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:05.974 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-04-26T19:51:05.975 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-04-26T19:51:05.975 INFO:teuthology.orchestra.run.smithi079.stdout:invalid new device string format 2024-04-26T19:51:05.975 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:05.975 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:05.975 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790" 2024-04-26T19:51:05.975 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790:/ 2024-04-26T19:51:05.975 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs,key=0,fsid=b372f41f-f07b-4764-9bef-bcff62f9a5d2" will pass to kernel 2024-04-26T19:51:05.975 INFO:teuthology.orchestra.run.smithi079.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T19:51:05.975 INFO:teuthology.orchestra.run.smithi079.stdout: You just mounted a file system that supports labels which does not 2024-04-26T19:51:05.976 INFO:teuthology.orchestra.run.smithi079.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T19:51:05.976 INFO:teuthology.orchestra.run.smithi079.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T19:51:05.976 INFO:teuthology.orchestra.run.smithi079.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T19:51:05.977 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T19:51:05.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:05.977 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T19:51:06.042 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:06.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:06.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:06.138 INFO:teuthology.orchestra.run.smithi079.stdout:client4571 2024-04-26T19:51:06.139 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:51:06.139 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:06.155 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:06.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:06.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:06.251 INFO:teuthology.orchestra.run.smithi079.stdout:client4571 2024-04-26T19:51:06.252 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:06.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:06.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:06.306 INFO:teuthology.orchestra.run.smithi079.stdout:client4571 2024-04-26T19:51:06.306 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b372f41f-f07b-4764-9bef-bcff62f9a5d2.client4571/status') 2024-04-26T19:51:06.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:06.363 INFO:teuthology.orchestra.run.smithi079.stdout:instance: client.4571 (3)172.21.15.79:0/3905225218 2024-04-26T19:51:06.364 INFO:teuthology.orchestra.run.smithi079.stdout:blocklisted: false 2024-04-26T19:51:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2024-04-26T19:51:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2024-04-26T19:51:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:70 bytes copied, 2.3997e-05 s, 2.9 MB/s 2024-04-26T19:51:06.364 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:06.370 INFO:tasks.kclient:client.2 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v1'} 2024-04-26T19:51:06.371 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T19:51:06.371 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T19:51:06.371 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T19:51:06.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.467+0000 7f6c981c1640 1 Processor -- start 2024-04-26T19:51:06.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.467+0000 7f6c981c1640 1 -- start start 2024-04-26T19:51:06.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c981c1640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6c900b4220 0x7f6c901630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:06.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c981c1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6c90160770 0x7f6c90163610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:06.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c981c1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c90144ca0 con 0x7f6c90160390 2024-04-26T19:51:06.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c981c1640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f6c90163b50 con 0x7f6c900b4220 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c981c1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f6c90163e00 con 0x7f6c90160770 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c9575c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6c900b4220 0x7f6c901630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c95f5d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6c90160390 0x7f6c900b3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53166/0 (socket says 172.21.15.79:53166) 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c95f5d640 1 -- 172.21.15.79:0/2517096795 learned_addr learned my addr 172.21.15.79:0/2517096795 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c9575c640 1 -- 172.21.15.79:0/2517096795 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6c90160770 msgr2=0x7f6c90163610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c9575c640 1 --2- 172.21.15.79:0/2517096795 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6c90160770 0x7f6c90163610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c9575c640 1 -- 172.21.15.79:0/2517096795 >> v1:172.21.15.79:6789/0 conn(0x7f6c90160390 legacy=0x7f6c900b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c94f5b640 1 -- 172.21.15.79:0/2517096795 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 909038900 0 0) 0x7f6c90144ca0 con 0x7f6c90160390 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.468+0000 7f6c9575c640 1 -- 172.21.15.79:0/2517096795 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c90144f30 con 0x7f6c900b4220 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c9575c640 1 --2- 172.21.15.79:0/2517096795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c900b4220 0x7f6c901630d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0097a0 tx=0x7f6c8c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454482f414d2042e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:06.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c94f5b640 1 -- 172.21.15.79:0/2517096795 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c037070 con 0x7f6c900b4220 2024-04-26T19:51:06.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c94f5b640 1 -- 172.21.15.79:0/2517096795 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c02ae50 con 0x7f6c900b4220 2024-04-26T19:51:06.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c94f5b640 1 -- 172.21.15.79:0/2517096795 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c032660 con 0x7f6c900b4220 2024-04-26T19:51:06.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c981c1640 1 -- 172.21.15.79:0/2517096795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c900b4220 msgr2=0x7f6c901630d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:06.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c981c1640 1 --2- 172.21.15.79:0/2517096795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c900b4220 0x7f6c901630d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0097a0 tx=0x7f6c8c02a100 comp rx=0 tx=0).stop 2024-04-26T19:51:06.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c981c1640 1 -- 172.21.15.79:0/2517096795 shutdown_connections 2024-04-26T19:51:06.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c981c1640 1 --2- 172.21.15.79:0/2517096795 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6c90160770 0x7f6c90163610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c981c1640 1 --2- 172.21.15.79:0/2517096795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c900b4220 0x7f6c901630d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c981c1640 1 -- 172.21.15.79:0/2517096795 >> 172.21.15.79:0/2517096795 conn(0x7f6c900b2720 msgr2=0x7f6c90070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c981c1640 1 -- 172.21.15.79:0/2517096795 shutdown_connections 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.469+0000 7f6c981c1640 1 -- 172.21.15.79:0/2517096795 wait complete. 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c981c1640 1 Processor -- start 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c981c1640 1 -- start start 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c981c1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6c900b4220 0x7f6c90158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c981c1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c90160390 0x7f6c9015c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c981c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90160770 0x7f6c900700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c981c1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6c90145a10 con 0x7f6c90160390 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c981c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6c901454f0 con 0x7f6c90160770 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c981c1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f6c90145b90 con 0x7f6c900b4220 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c9575c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c90160390 0x7f6c9015c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:06.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c9575c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c90160390 0x7f6c9015c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46572/0 (socket says 172.21.15.79:46572) 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.471+0000 7f6c9575c640 1 -- 172.21.15.79:0/3253962870 learned_addr learned my addr 172.21.15.79:0/3253962870 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c9575c640 1 -- 172.21.15.79:0/3253962870 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6c900b4220 msgr2=0x7f6c90158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c95f5d640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6c900b4220 0x7f6c90158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c9675e640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90160770 0x7f6c900700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c9575c640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6c900b4220 0x7f6c90158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c9575c640 1 -- 172.21.15.79:0/3253962870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90160770 msgr2=0x7f6c900700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c9575c640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90160770 0x7f6c900700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c9575c640 1 -- 172.21.15.79:0/3253962870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c900b1630 con 0x7f6c90160390 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c9575c640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c90160390 0x7f6c9015c870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0097a0 tx=0x7f6c8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c827fc640 1 -- 172.21.15.79:0/3253962870 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c037040 con 0x7f6c90160390 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c827fc640 1 -- 172.21.15.79:0/3253962870 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c004030 con 0x7f6c90160390 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c827fc640 1 -- 172.21.15.79:0/3253962870 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c006e10 con 0x7f6c90160390 2024-04-26T19:51:06.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c900b1910 con 0x7f6c90160390 2024-04-26T19:51:06.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.472+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c9015cdb0 con 0x7f6c90160390 2024-04-26T19:51:06.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.473+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c900506c0 con 0x7f6c90160390 2024-04-26T19:51:06.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.473+0000 7f6c827fc640 1 -- 172.21.15.79:0/3253962870 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c8c03d650 con 0x7f6c90160390 2024-04-26T19:51:06.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.474+0000 7f6c827fc640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6c7406c8f0 0x7f6c7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:06.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.474+0000 7f6c95f5d640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6c7406c8f0 0x7f6c7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:06.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.474+0000 7f6c827fc640 1 -- 172.21.15.79:0/3253962870 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6c8c0ac6f0 con 0x7f6c90160390 2024-04-26T19:51:06.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.474+0000 7f6c95f5d640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6c7406c8f0 0x7f6c7406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000f60 tx=0x7f6c84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:06.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.478+0000 7f6c827fc640 1 -- 172.21.15.79:0/3253962870 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6c8c042020 con 0x7f6c90160390 2024-04-26T19:51:06.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.693+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6c9006d7b0 con 0x7f6c90160390 2024-04-26T19:51:06.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.693+0000 7f6c827fc640 1 -- 172.21.15.79:0/3253962870 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f6c8c07a870 con 0x7f6c90160390 2024-04-26T19:51:06.694 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T19:51:06.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6c7406c8f0 msgr2=0x7f6c7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:06.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6c7406c8f0 0x7f6c7406edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6c84000f60 tx=0x7f6c84009040 comp rx=0 tx=0).stop 2024-04-26T19:51:06.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c90160390 msgr2=0x7f6c9015c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c90160390 0x7f6c9015c870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0097a0 tx=0x7f6c8c0046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 shutdown_connections 2024-04-26T19:51:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6c7406c8f0 0x7f6c7406edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6c90160770 0x7f6c900700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6c90160390 0x7f6c9015c870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 --2- 172.21.15.79:0/3253962870 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6c900b4220 0x7f6c90158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 >> 172.21.15.79:0/3253962870 conn(0x7f6c900b2720 msgr2=0x7f6c900b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 shutdown_connections 2024-04-26T19:51:06.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06.695+0000 7f6c981c1640 1 -- 172.21.15.79:0/3253962870 wait complete. 2024-04-26T19:51:06.707 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T19:51:06.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T19:51:06.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2024-04-26T19:51:06.707 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-26T19:51:06.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-26T19:51:06.708 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T19:51:06.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T19:51:06.708 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T19:51:06.708 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-26T19:51:06.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:06.708 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2024-04-26T19:51:06.723 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:51:06.723 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f0 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f1 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:06.724 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42682sec preferred_lft 42682sec 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::5909:6a14:c050:be1a/64 scope link noprefixroute 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 8a:10:f3:c9:4a:e1 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::347f:e9ff:fe5a:5029/64 scope link 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 8a:10:f3:c9:4a:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::8810:f3ff:fec9:4ae1/64 scope link tentative 2024-04-26T19:51:06.725 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:06.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:06.726 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T19:51:06.785 INFO:teuthology.orchestra.run.smithi079.stdout:1 2024-04-26T19:51:06.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:06.787 DEBUG:teuthology.orchestra.run.smithi079:> route 2024-04-26T19:51:06.843 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2024-04-26T19:51:06.843 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T19:51:06.843 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T19:51:06.843 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T19:51:06.844 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T19:51:06.844 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:06.844 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:06.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T19:51:06.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T19:51:06.844 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T19:51:06.844 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:06.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:06.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:06.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:06.966 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-04-26T19:51:06.981 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T19:51:06.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:06.982 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list-id 2024-04-26T19:51:07.036 INFO:teuthology.orchestra.run.smithi079.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-26T19:51:07.037 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:07.037 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:07.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T19:51:07.037 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-26T19:51:07.037 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:07.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:07.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:07.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:07.152 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T19:51:07.175 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:51:07.175 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:51:07.175 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:51:07.175 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:07.175 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2024-04-26T19:51:07.176 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:07.176 INFO:teuthology.orchestra.run.smithi079.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T19:51:07.176 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether d2:90:30:86:63:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T19:51:07.176 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T19:51:07.176 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:07.176 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::d090:30ff:fe86:6322/64 scope link tentative 2024-04-26T19:51:07.176 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:07.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:07.177 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-26T19:51:07.240 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:51:07.240 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether d2:90:30:86:63:22 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::d090:30ff:fe86:6322/64 scope link tentative 2024-04-26T19:51:07.241 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:07.242 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-26T19:51:07.243 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:07.243 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:07.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-26T19:51:07.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T19:51:07.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-26T19:51:07.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-26T19:51:07.243 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-26T19:51:07.243 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:07.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:07.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:07.389 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:07.389 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:07.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.1 up 2024-04-26T19:51:07.389 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T19:51:07.389 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:07.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:07.457 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:07.464 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:07.464 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-26T19:51:07.479 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-26T19:51:07.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:07.479 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-26T19:51:07.534 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T19:51:07.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:07.534 DEBUG:teuthology.orchestra.run.smithi079:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs 2024-04-26T19:51:07.615 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs 2024-04-26T19:51:07.615 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs". 2024-04-26T19:51:07.615 INFO:teuthology.orchestra.run.smithi079.stdout:invalid new device string format 2024-04-26T19:51:07.624 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs 2024-04-26T19:51:07.624 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs". 2024-04-26T19:51:07.624 INFO:teuthology.orchestra.run.smithi079.stdout:invalid new device string format 2024-04-26T19:51:07.624 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:07.624 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:07.629 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs 2024-04-26T19:51:07.629 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs". 2024-04-26T19:51:07.629 INFO:teuthology.orchestra.run.smithi079.stdout:invalid new device string format 2024-04-26T19:51:07.629 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:07.629 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:07.629 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790" 2024-04-26T19:51:07.630 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790:/ 2024-04-26T19:51:07.630 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=2,mds_namespace=cephfs,key=2,fsid=b372f41f-f07b-4764-9bef-bcff62f9a5d2" will pass to kernel 2024-04-26T19:51:07.630 INFO:teuthology.orchestra.run.smithi079.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-26T19:51:07.630 INFO:teuthology.orchestra.run.smithi079.stdout: You just mounted a file system that supports labels which does not 2024-04-26T19:51:07.630 INFO:teuthology.orchestra.run.smithi079.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T19:51:07.630 INFO:teuthology.orchestra.run.smithi079.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T19:51:07.630 INFO:teuthology.orchestra.run.smithi079.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T19:51:07.631 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T19:51:07.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:07.632 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-26T19:51:07.696 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:07.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:07.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:07.793 INFO:teuthology.orchestra.run.smithi079.stdout:client4753 2024-04-26T19:51:07.793 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:51:07.793 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:07.809 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:07.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:07.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:07.904 INFO:teuthology.orchestra.run.smithi079.stdout:client4753 2024-04-26T19:51:07.905 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:07.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:07.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:07.960 INFO:teuthology.orchestra.run.smithi079.stdout:client4753 2024-04-26T19:51:07.960 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b372f41f-f07b-4764-9bef-bcff62f9a5d2.client4753/status') 2024-04-26T19:51:08.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:08.017 INFO:teuthology.orchestra.run.smithi079.stdout:instance: client.4753 (3)172.21.15.79:0/1377167565 2024-04-26T19:51:08.018 INFO:teuthology.orchestra.run.smithi079.stdout:blocklisted: false 2024-04-26T19:51:08.018 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records in 2024-04-26T19:51:08.018 INFO:teuthology.orchestra.run.smithi079.stderr:0+1 records out 2024-04-26T19:51:08.018 INFO:teuthology.orchestra.run.smithi079.stderr:70 bytes copied, 2.1724e-05 s, 3.2 MB/s 2024-04-26T19:51:08.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:08.024 INFO:tasks.kclient:client.1 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v1'} 2024-04-26T19:51:08.024 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T19:51:08.024 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T19:51:08.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T19:51:08.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.120+0000 7fc9f9fda640 1 Processor -- start 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.120+0000 7fc9f9fda640 1 -- start start 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.121+0000 7fc9f9fda640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc9f40b4250 0x7fc9f4163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.121+0000 7fc9f9fda640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f4160280 con 0x7fc9f4159c70 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.121+0000 7fc9f9fda640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9f4160560 con 0x7fc9f4159890 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.121+0000 7fc9f9fda640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc9f4160840 con 0x7fc9f40b4250 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.121+0000 7fc9f2ffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc9f4159c70 0x7fc9f40b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35188/0 (socket says 172.21.15.79:35188) 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.121+0000 7fc9f2ffd640 1 -- 172.21.15.79:0/632603584 learned_addr learned my addr 172.21.15.79:0/632603584 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1134796159 0 0) 0x7fc9f4160280 con 0x7fc9f4159c70 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f3fff640 1 --2- 172.21.15.79:0/632603584 >> v2:172.21.15.83:3300/0 conn(0x7fc9f40b4250 0x7fc9f4163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:08.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9dc003520 con 0x7fc9f4159c70 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 939764025 0 0) 0x7fc9dc003520 con 0x7fc9f4159c70 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 >> v2:172.21.15.83:3300/0 conn(0x7fc9f40b4250 msgr2=0x7fc9f4163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 --2- 172.21.15.79:0/632603584 >> v2:172.21.15.83:3300/0 conn(0x7fc9f40b4250 0x7fc9f4163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 >> v1:172.21.15.83:6789/0 conn(0x7fc9f4159890 legacy=0x7fc9f415fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f4160e20 con 0x7fc9f4159c70 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fc9e8001670 con 0x7fc9f4159c70 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 >> v1:172.21.15.79:6789/0 conn(0x7fc9f4159c70 legacy=0x7fc9f40b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc9dc0039b0 0x7fc9dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9dc0051c0 0x7fc9dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4160840 con 0x7fc9dc004800 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc00c160 con 0x7fc9dc0051c0 2024-04-26T19:51:08.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc9dc00c360 con 0x7fc9dc0039b0 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f2ffd640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f3fff640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9dc0051c0 0x7fc9dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f37fe640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc9dc0039b0 0x7fc9dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f2ffd640 1 -- 172.21.15.79:0/632603584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc9dc0039b0 msgr2=0x7fc9dc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f2ffd640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc9dc0039b0 0x7fc9dc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f2ffd640 1 -- 172.21.15.79:0/632603584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9dc0051c0 msgr2=0x7fc9dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f2ffd640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9dc0051c0 0x7fc9dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f3fff640 1 -- 172.21.15.79:0/632603584 reap_dead start 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f2ffd640 1 -- 172.21.15.79:0/632603584 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc003520 con 0x7fc9dc004800 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.122+0000 7fc9f37fe640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc9dc0039b0 0x7fc9dc003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.123+0000 7fc9f2ffd640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9dc004c80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c240 tx=0x7fc9dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196a7d5756ec5d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.123+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8007d10 con 0x7fc9dc004800 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.123+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e8005ad0 con 0x7fc9dc004800 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.123+0000 7fc9f27fc640 1 -- 172.21.15.79:0/632603584 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8007070 con 0x7fc9dc004800 2024-04-26T19:51:08.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.124+0000 7fc9f9fda640 1 -- 172.21.15.79:0/632603584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 msgr2=0x7fc9dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:08.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.124+0000 7fc9f9fda640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9dc004c80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c240 tx=0x7fc9dc005830 comp rx=0 tx=0).stop 2024-04-26T19:51:08.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.124+0000 7fc9f9fda640 1 -- 172.21.15.79:0/632603584 shutdown_connections 2024-04-26T19:51:08.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.124+0000 7fc9f9fda640 1 --2- 172.21.15.79:0/632603584 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9dc004c80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.124+0000 7fc9f9fda640 1 -- 172.21.15.79:0/632603584 >> 172.21.15.79:0/632603584 conn(0x7fc9f40b2720 msgr2=0x7fc9f4070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:08.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.124+0000 7fc9f9fda640 1 -- 172.21.15.79:0/632603584 shutdown_connections 2024-04-26T19:51:08.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.124+0000 7fc9f9fda640 1 -- 172.21.15.79:0/632603584 wait complete. 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.125+0000 7fc9f9fda640 1 Processor -- start 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.125+0000 7fc9f9fda640 1 -- start start 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f9fda640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9f414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f9fda640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9f41f5950 0x7fc9f415c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f9fda640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc9f415c720 0x7fc9f41f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f9fda640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4163bd0 con 0x7fc9dc004800 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f9fda640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f4163a50 con 0x7fc9f41f5950 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f9fda640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc9f4163d50 con 0x7fc9f415c720 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9f414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9f414e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44130/0 (socket says 172.21.15.79:44130) 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f37fe640 1 -- 172.21.15.79:0/1285041817 learned_addr learned my addr 172.21.15.79:0/1285041817 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:08.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f37fe640 1 -- 172.21.15.79:0/1285041817 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc9f415c720 msgr2=0x7fc9f41f84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f37fe640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc9f415c720 0x7fc9f41f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f37fe640 1 -- 172.21.15.79:0/1285041817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9f41f5950 msgr2=0x7fc9f415c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f37fe640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9f41f5950 0x7fc9f415c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f37fe640 1 -- 172.21.15.79:0/1285041817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f415b570 con 0x7fc9dc004800 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f37fe640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9f414e4a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400e400 tx=0x7fc9e400e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9dbfff640 1 -- 172.21.15.79:0/1285041817 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4019070 con 0x7fc9dc004800 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9dbfff640 1 -- 172.21.15.79:0/1285041817 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e40052e0 con 0x7fc9dc004800 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f415a4d0 con 0x7fc9dc004800 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9dbfff640 1 -- 172.21.15.79:0/1285041817 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4003830 con 0x7fc9dc004800 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.126+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f415aa90 con 0x7fc9dc004800 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.127+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9f415cb40 con 0x7fc9dc004800 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.128+0000 7fc9dbfff640 1 -- 172.21.15.79:0/1285041817 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc9e401c070 con 0x7fc9dc004800 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.128+0000 7fc9dbfff640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc9e006c8f0 0x7fc9e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:08.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.128+0000 7fc9f2ffd640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc9e006c8f0 0x7fc9e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:08.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.128+0000 7fc9dbfff640 1 -- 172.21.15.79:0/1285041817 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc9e408b2e0 con 0x7fc9dc004800 2024-04-26T19:51:08.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.129+0000 7fc9f2ffd640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc9e006c8f0 0x7fc9e006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8032000 tx=0x7fc9e8009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:08.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.133+0000 7fc9dbfff640 1 -- 172.21.15.79:0/1285041817 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc9e4058d80 con 0x7fc9dc004800 2024-04-26T19:51:08.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.351+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc9f40b1ad0 con 0x7fc9dc004800 2024-04-26T19:51:08.352 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.351+0000 7fc9dbfff640 1 -- 172.21.15.79:0/1285041817 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fc9f40b1ad0 con 0x7fc9dc004800 2024-04-26T19:51:08.352 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T19:51:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc9e006c8f0 msgr2=0x7fc9e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc9e006c8f0 0x7fc9e006edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8032000 tx=0x7fc9e8009030 comp rx=0 tx=0).stop 2024-04-26T19:51:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 msgr2=0x7fc9f414e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9f414e4a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9e400e400 tx=0x7fc9e400e8c0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 shutdown_connections 2024-04-26T19:51:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc9e006c8f0 0x7fc9e006edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc9f415c720 0x7fc9f41f84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc9f41f5950 0x7fc9f415c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 --2- 172.21.15.79:0/1285041817 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc9dc004800 0x7fc9f414e4a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 >> 172.21.15.79:0/1285041817 conn(0x7fc9f40b2720 msgr2=0x7fc9f415f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 shutdown_connections 2024-04-26T19:51:08.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:08.353+0000 7fc9f9fda640 1 -- 172.21.15.79:0/1285041817 wait complete. 2024-04-26T19:51:08.365 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T19:51:08.365 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T19:51:08.365 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-26T19:51:08.365 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T19:51:08.365 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T19:51:08.365 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T19:51:08.365 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T19:51:08.365 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T19:51:08.365 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T19:51:08.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:08.365 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-26T19:51:08.384 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:51:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:51:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:51:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-26T19:51:08.384 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42592sec preferred_lft 42592sec 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::25a0:d41e:df4d:114a/64 scope link noprefixroute 2024-04-26T19:51:08.385 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:08.386 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T19:51:08.386 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:08.386 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-26T19:51:08.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add name ceph-brx type bridge 2024-04-26T19:51:08.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr flush dev ceph-brx 2024-04-26T19:51:08.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set ceph-brx up 2024-04-26T19:51:08.386 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T19:51:08.386 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-26T19:51:08.479 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:08.857 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:08.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:08.867 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T19:51:08.891 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-26T19:51:08.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:08.892 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-26T19:51:08.957 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-26T19:51:08.957 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T19:51:08.957 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T19:51:08.957 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T19:51:08.957 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T19:51:08.958 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:08.958 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-26T19:51:08.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T19:51:08.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T19:51:08.958 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T19:51:08.958 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-26T19:51:09.046 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:09.209 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:09.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:09.217 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-26T19:51:09.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:09.232 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-26T19:51:09.287 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:09.287 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-26T19:51:09.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T19:51:09.288 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T19:51:09.288 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-26T19:51:09.378 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:09.399 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:09.405 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T19:51:09.405 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:09.406 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-26T19:51:09.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T19:51:09.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T19:51:09.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T19:51:09.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T19:51:09.406 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T19:51:09.406 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-26T19:51:09.453 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:09.522 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:09.528 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:09.528 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-26T19:51:09.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.0 up 2024-04-26T19:51:09.528 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T19:51:09.529 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-26T19:51:09.576 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:09.597 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:09.603 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:09.603 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T19:51:09.617 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T19:51:09.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:09.617 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T19:51:09.671 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T19:51:09.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:09.671 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs 2024-04-26T19:51:09.761 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs 2024-04-26T19:51:09.761 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs". 2024-04-26T19:51:09.761 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-26T19:51:09.869 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs 2024-04-26T19:51:09.869 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs". 2024-04-26T19:51:09.869 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-26T19:51:09.870 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:09.870 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:09.874 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs 2024-04-26T19:51:09.874 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs". 2024-04-26T19:51:09.874 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-26T19:51:09.874 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:09.874 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:09.874 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790" 2024-04-26T19:51:09.875 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790:/ 2024-04-26T19:51:09.875 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=1,mds_namespace=cephfs,key=1,fsid=b372f41f-f07b-4764-9bef-bcff62f9a5d2" will pass to kernel 2024-04-26T19:51:09.875 INFO:teuthology.orchestra.run.smithi083.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T19:51:09.875 INFO:teuthology.orchestra.run.smithi083.stdout: You just mounted a file system that supports labels which does not 2024-04-26T19:51:09.875 INFO:teuthology.orchestra.run.smithi083.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T19:51:09.875 INFO:teuthology.orchestra.run.smithi083.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T19:51:09.875 INFO:teuthology.orchestra.run.smithi083.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T19:51:09.877 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T19:51:09.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:09.877 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T19:51:09.901 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:09.989 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:09.990 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:09.996 INFO:teuthology.orchestra.run.smithi083.stdout:client4574 2024-04-26T19:51:09.996 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:51:09.997 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:10.012 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:10.100 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:10.102 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:10.108 INFO:teuthology.orchestra.run.smithi083.stdout:client4574 2024-04-26T19:51:10.108 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:10.155 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:10.156 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:10.162 INFO:teuthology.orchestra.run.smithi083.stdout:client4574 2024-04-26T19:51:10.163 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b372f41f-f07b-4764-9bef-bcff62f9a5d2.client4574/status') 2024-04-26T19:51:10.211 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:10.221 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.4574 (3)192.168.0.1:0/1653028032 2024-04-26T19:51:10.221 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2024-04-26T19:51:10.221 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2024-04-26T19:51:10.222 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2024-04-26T19:51:10.222 INFO:teuthology.orchestra.run.smithi083.stderr:69 bytes copied, 2.2226e-05 s, 3.1 MB/s 2024-04-26T19:51:10.222 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:10.229 INFO:tasks.kclient:client.3 config is {'snapdirname': '.customsnapkernel', 'syntax': 'v1'} 2024-04-26T19:51:10.230 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T19:51:10.230 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T19:51:10.230 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T19:51:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.337+0000 7f608892b640 1 Processor -- start 2024-04-26T19:51:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.337+0000 7f608892b640 1 -- start start 2024-04-26T19:51:10.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.338+0000 7f608892b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f608406fb20 0x7f60840d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.338+0000 7f608892b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6084148fd0 0x7f60840d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.338+0000 7f608892b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60840b1290 con 0x7f608406ff00 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.338+0000 7f608892b640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f60840d3e60 con 0x7f608406fb20 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.338+0000 7f608892b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f60840d4090 con 0x7f6084148fd0 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.338+0000 7f608259c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f608406fb20 0x7f60840d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.338+0000 7f608259c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f608406fb20 0x7f60840d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44140/0 (socket says 172.21.15.79:44140) 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.338+0000 7f608259c640 1 -- 172.21.15.79:0/3408040365 learned_addr learned my addr 172.21.15.79:0/3408040365 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.339+0000 7f6082d9d640 1 --2- 172.21.15.79:0/3408040365 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6084148fd0 0x7f60840d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.339+0000 7f608259c640 1 -- 172.21.15.79:0/3408040365 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6084148fd0 msgr2=0x7f60840d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.339+0000 7f608259c640 1 --2- 172.21.15.79:0/3408040365 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6084148fd0 0x7f60840d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.339+0000 7f608259c640 1 -- 172.21.15.79:0/3408040365 >> v1:172.21.15.83:6789/0 conn(0x7f608406ff00 legacy=0x7f60841488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:10.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.339+0000 7f608259c640 1 -- 172.21.15.79:0/3408040365 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60840d4370 con 0x7f608406fb20 2024-04-26T19:51:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.339+0000 7f608259c640 1 --2- 172.21.15.79:0/3408040365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f608406fb20 0x7f60840d2cd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60740098a0 tx=0x7f607402a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0fcdd968e93160 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608159a640 1 -- 172.21.15.79:0/3408040365 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074037070 con 0x7f608406fb20 2024-04-26T19:51:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608159a640 1 -- 172.21.15.79:0/3408040365 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6074032640 con 0x7f608406fb20 2024-04-26T19:51:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608159a640 1 -- 172.21.15.79:0/3408040365 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074032a10 con 0x7f608406fb20 2024-04-26T19:51:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608892b640 1 -- 172.21.15.79:0/3408040365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f608406fb20 msgr2=0x7f60840d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608892b640 1 --2- 172.21.15.79:0/3408040365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f608406fb20 0x7f60840d2cd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f60740098a0 tx=0x7f607402a320 comp rx=0 tx=0).stop 2024-04-26T19:51:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608892b640 1 -- 172.21.15.79:0/3408040365 shutdown_connections 2024-04-26T19:51:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608892b640 1 --2- 172.21.15.79:0/3408040365 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6084148fd0 0x7f60840d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608892b640 1 --2- 172.21.15.79:0/3408040365 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f608406fb20 0x7f60840d2cd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608892b640 1 -- 172.21.15.79:0/3408040365 >> 172.21.15.79:0/3408040365 conn(0x7f6084071c30 msgr2=0x7f608406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608892b640 1 -- 172.21.15.79:0/3408040365 shutdown_connections 2024-04-26T19:51:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.340+0000 7f608892b640 1 -- 172.21.15.79:0/3408040365 wait complete. 2024-04-26T19:51:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.342+0000 7f608892b640 1 Processor -- start 2024-04-26T19:51:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.342+0000 7f608892b640 1 -- start start 2024-04-26T19:51:10.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.342+0000 7f608892b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f608406fb20 0x7f608415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:10.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.342+0000 7f608892b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f608406ff00 0x7f6084145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:10.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.342+0000 7f608892b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6084148fd0 0x7f608413aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:10.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.342+0000 7f608892b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f60840d4c10 con 0x7f6084148fd0 2024-04-26T19:51:10.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.342+0000 7f608892b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f60840d4930 con 0x7f608406fb20 2024-04-26T19:51:10.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.342+0000 7f608892b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f60840aebb0 con 0x7f608406ff00 2024-04-26T19:51:10.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6082d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6084148fd0 0x7f608413aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:10.346 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6082d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6084148fd0 0x7f608413aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44146/0 (socket says 172.21.15.79:44146) 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6082d9d640 1 -- 172.21.15.79:0/3780143281 learned_addr learned my addr 172.21.15.79:0/3780143281 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f608259c640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f608406fb20 0x7f608415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6081d9b640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f608406ff00 0x7f6084145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6082d9d640 1 -- 172.21.15.79:0/3780143281 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f608406ff00 msgr2=0x7f6084145e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6082d9d640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f608406ff00 0x7f6084145e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6082d9d640 1 -- 172.21.15.79:0/3780143281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f608406fb20 msgr2=0x7f608415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6082d9d640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f608406fb20 0x7f608415ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6082d9d640 1 -- 172.21.15.79:0/3780143281 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6084144df0 con 0x7f6084148fd0 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6082d9d640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6084148fd0 0x7f608413aaa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f607c00efc0 tx=0x7f607c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6072ffd640 1 -- 172.21.15.79:0/3780143281 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c018020 con 0x7f6084148fd0 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6072ffd640 1 -- 172.21.15.79:0/3780143281 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f607c014480 con 0x7f6084148fd0 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60841450d0 con 0x7f6084148fd0 2024-04-26T19:51:10.347 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f6072ffd640 1 -- 172.21.15.79:0/3780143281 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607c014870 con 0x7f6084148fd0 2024-04-26T19:51:10.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.343+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6084145690 con 0x7f6084148fd0 2024-04-26T19:51:10.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.344+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60840506c0 con 0x7f6084148fd0 2024-04-26T19:51:10.348 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.345+0000 7f6072ffd640 1 -- 172.21.15.79:0/3780143281 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f607c005d10 con 0x7f6084148fd0 2024-04-26T19:51:10.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.345+0000 7f6072ffd640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f606406c8f0 0x7f606406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:10.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.345+0000 7f6072ffd640 1 -- 172.21.15.79:0/3780143281 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f607c08c780 con 0x7f6084148fd0 2024-04-26T19:51:10.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.345+0000 7f608259c640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f606406c8f0 0x7f606406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:10.349 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.346+0000 7f608259c640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f606406c8f0 0x7f606406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6074034000 tx=0x7f607402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:10.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.349+0000 7f6072ffd640 1 -- 172.21.15.79:0/3780143281 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f607c022070 con 0x7f6084148fd0 2024-04-26T19:51:10.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.561+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f60840d3700 con 0x7f6084148fd0 2024-04-26T19:51:10.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.561+0000 7f6072ffd640 1 -- 172.21.15.79:0/3780143281 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f607c010040 con 0x7f6084148fd0 2024-04-26T19:51:10.562 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-26T19:51:10.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f606406c8f0 msgr2=0x7f606406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f606406c8f0 0x7f606406edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6074034000 tx=0x7f607402e000 comp rx=0 tx=0).stop 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6084148fd0 msgr2=0x7f608413aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6084148fd0 0x7f608413aaa0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f607c00efc0 tx=0x7f607c00c490 comp rx=0 tx=0).stop 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 shutdown_connections 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f606406c8f0 0x7f606406edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6084148fd0 0x7f608413aaa0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f608406ff00 0x7f6084145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 --2- 172.21.15.79:0/3780143281 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f608406fb20 0x7f608415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 >> 172.21.15.79:0/3780143281 conn(0x7f6084071c30 msgr2=0x7f60840b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 shutdown_connections 2024-04-26T19:51:10.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:10.563+0000 7f608892b640 1 -- 172.21.15.79:0/3780143281 wait complete. 2024-04-26T19:51:10.574 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T19:51:10.574 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T19:51:10.574 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi083.front.sepia.ceph.com 2024-04-26T19:51:10.574 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-26T19:51:10.575 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-26T19:51:10.575 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T19:51:10.575 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T19:51:10.575 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T19:51:10.575 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-26T19:51:10.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:10.575 DEBUG:teuthology.orchestra.run.smithi083:> ip addr 2024-04-26T19:51:10.591 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:51:10.591 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:51:10.591 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:51:10.591 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:10.591 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-26T19:51:10.591 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:10.591 INFO:teuthology.orchestra.run.smithi083.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:10.591 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1e brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f0 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:d8:da:1f brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: altname enp7s0f1 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:90 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 0c:c4:7a:8f:ce:91 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: inet 172.21.15.83/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft 42590sec preferred_lft 42590sec 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::25a0:d41e:df4d:114a/64 scope link noprefixroute 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 52:8e:a3:f7:41:44 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:10.592 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T19:51:10.593 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:10.593 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::6836:1bff:fe48:a01b/64 scope link 2024-04-26T19:51:10.593 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:10.593 INFO:teuthology.orchestra.run.smithi083.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T19:51:10.593 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether 52:8e:a3:f7:41:44 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T19:51:10.593 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::508e:a3ff:fef7:4144/64 scope link tentative 2024-04-26T19:51:10.593 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:10.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:10.593 DEBUG:teuthology.orchestra.run.smithi083:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T19:51:10.653 INFO:teuthology.orchestra.run.smithi083.stdout:1 2024-04-26T19:51:10.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:10.655 DEBUG:teuthology.orchestra.run.smithi083:> route 2024-04-26T19:51:10.710 INFO:teuthology.orchestra.run.smithi083.stdout:Kernel IP routing table 2024-04-26T19:51:10.710 INFO:teuthology.orchestra.run.smithi083.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T19:51:10.711 INFO:teuthology.orchestra.run.smithi083.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T19:51:10.711 INFO:teuthology.orchestra.run.smithi083.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T19:51:10.711 INFO:teuthology.orchestra.run.smithi083.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T19:51:10.711 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:10.711 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-26T19:51:10.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T19:51:10.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T19:51:10.711 DEBUG:teuthology.orchestra.run.smithi083:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T19:51:10.711 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-26T19:51:10.798 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:10.828 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:10.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:10.834 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-26T19:51:10.849 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T19:51:10.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:10.849 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list-id 2024-04-26T19:51:10.904 INFO:teuthology.orchestra.run.smithi083.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-26T19:51:10.904 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:10.904 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-26T19:51:10.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T19:51:10.904 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-26T19:51:10.904 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-26T19:51:10.992 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:11.012 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:11.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:11.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T19:51:11.041 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:51:11.041 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:51:11.041 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:51:11.041 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:11.042 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-26T19:51:11.042 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:11.042 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T19:51:11.042 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether b6:d2:f7:34:ea:36 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T19:51:11.042 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T19:51:11.042 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:11.042 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::b4d2:f7ff:fe34:ea36/64 scope link tentative 2024-04-26T19:51:11.042 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:11.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:11.043 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-26T19:51:11.106 INFO:teuthology.orchestra.run.smithi083.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:51:11.106 INFO:teuthology.orchestra.run.smithi083.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:51:11.106 INFO:teuthology.orchestra.run.smithi083.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:51:11.106 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:11.107 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 ::1/128 scope host 2024-04-26T19:51:11.107 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:11.107 INFO:teuthology.orchestra.run.smithi083.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T19:51:11.107 INFO:teuthology.orchestra.run.smithi083.stdout: link/ether b6:d2:f7:34:ea:36 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-26T19:51:11.107 INFO:teuthology.orchestra.run.smithi083.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-26T19:51:11.107 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:11.107 INFO:teuthology.orchestra.run.smithi083.stdout: inet6 fe80::b4d2:f7ff:fe34:ea36/64 scope link 2024-04-26T19:51:11.107 INFO:teuthology.orchestra.run.smithi083.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:11.108 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-26T19:51:11.108 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:11.108 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-26T19:51:11.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-26T19:51:11.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-26T19:51:11.108 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-26T19:51:11.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-26T19:51:11.109 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-26T19:51:11.109 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-26T19:51:11.195 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:11.253 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:11.259 DEBUG:teuthology.orchestra.run.smithi083:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:11.259 DEBUG:teuthology.orchestra.run.smithi083:> set -e 2024-04-26T19:51:11.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set brx.1 up 2024-04-26T19:51:11.260 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link set dev brx.1 master ceph-brx 2024-04-26T19:51:11.260 DEBUG:teuthology.orchestra.run.smithi083:> ') 2024-04-26T19:51:11.307 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:11.328 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:11.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:11.334 DEBUG:teuthology.orchestra.run.smithi083:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-26T19:51:11.349 INFO:teuthology.orchestra.run.smithi083.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-26T19:51:11.349 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:11.349 DEBUG:teuthology.orchestra.run.smithi083:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-26T19:51:11.404 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T19:51:11.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:11.404 DEBUG:teuthology.orchestra.run.smithi083:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs 2024-04-26T19:51:11.485 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs 2024-04-26T19:51:11.485 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs". 2024-04-26T19:51:11.485 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-26T19:51:11.493 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs 2024-04-26T19:51:11.493 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs". 2024-04-26T19:51:11.493 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-26T19:51:11.494 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:11.494 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:11.498 INFO:teuthology.orchestra.run.smithi083.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs 2024-04-26T19:51:11.498 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs". 2024-04-26T19:51:11.498 INFO:teuthology.orchestra.run.smithi083.stdout:invalid new device string format 2024-04-26T19:51:11.499 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:11.499 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:11.499 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: resolved to: "172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790" 2024-04-26T19:51:11.499 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790:/ 2024-04-26T19:51:11.499 INFO:teuthology.orchestra.run.smithi083.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=3,mds_namespace=cephfs,key=3,fsid=b372f41f-f07b-4764-9bef-bcff62f9a5d2" will pass to kernel 2024-04-26T19:51:11.500 INFO:teuthology.orchestra.run.smithi083.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-26T19:51:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: You just mounted a file system that supports labels which does not 2024-04-26T19:51:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T19:51:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T19:51:11.500 INFO:teuthology.orchestra.run.smithi083.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T19:51:11.502 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T19:51:11.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:11.503 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-26T19:51:11.567 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:11.655 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:11.656 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:11.662 INFO:teuthology.orchestra.run.smithi083.stdout:client4768 2024-04-26T19:51:11.663 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:51:11.663 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:11.678 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:11.765 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:11.767 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:11.772 INFO:teuthology.orchestra.run.smithi083.stdout:client4768 2024-04-26T19:51:11.772 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:11.820 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:11.822 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:11.828 INFO:teuthology.orchestra.run.smithi083.stdout:client4768 2024-04-26T19:51:11.828 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b372f41f-f07b-4764-9bef-bcff62f9a5d2.client4768/status') 2024-04-26T19:51:11.876 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:11.885 INFO:teuthology.orchestra.run.smithi083.stdout:instance: client.4768 (3)192.168.0.2:0/4029707862 2024-04-26T19:51:11.885 INFO:teuthology.orchestra.run.smithi083.stdout:blocklisted: false 2024-04-26T19:51:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records in 2024-04-26T19:51:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:0+1 records out 2024-04-26T19:51:11.885 INFO:teuthology.orchestra.run.smithi083.stderr:69 bytes copied, 2.2125e-05 s, 3.1 MB/s 2024-04-26T19:51:11.886 INFO:teuthology.orchestra.run.smithi083.stderr:2024-04-26T19:51:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:11.892 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-26T19:51:11.908 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:12.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.017+0000 7f8aa5e93640 1 Processor -- start 2024-04-26T19:51:12.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.017+0000 7f8aa5e93640 1 -- start start 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8aa5e93640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa015d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8aa5e93640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8aa00b44f0 0x7f8aa00cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8aa5e93640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa015d670 con 0x7f8aa00cd9c0 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8aa5e93640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa0073110 con 0x7f8aa00b44f0 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8aa5e93640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa00ce1a0 con 0x7f8aa0156670 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9f7fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9ffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f8aa00b44f0 0x7f8aa00cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9f7fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa015d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:40454/0 (socket says 172.21.15.79:40454) 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9f7fe640 1 -- 172.21.15.79:0/3717874901 learned_addr learned my addr 172.21.15.79:0/3717874901 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9effd640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f8aa00cd9c0 0x7f8aa00b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:40000/0 (socket says 172.21.15.79:40000) 2024-04-26T19:51:12.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9f7fe640 1 -- 172.21.15.79:0/3717874901 >> v2:172.21.15.83:3300/0 conn(0x7f8aa00b44f0 msgr2=0x7f8aa00cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9f7fe640 1 --2- 172.21.15.79:0/3717874901 >> v2:172.21.15.83:3300/0 conn(0x7f8aa00b44f0 0x7f8aa00cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9f7fe640 1 -- 172.21.15.79:0/3717874901 >> v1:172.21.15.83:6789/0 conn(0x7f8aa00cd9c0 legacy=0x7f8aa00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9f7fe640 1 -- 172.21.15.79:0/3717874901 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa00cdec0 con 0x7f8aa0156670 2024-04-26T19:51:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.018+0000 7f8a9ffff640 1 --2- 172.21.15.79:0/3717874901 >> v2:172.21.15.83:3300/0 conn(0x7f8aa00b44f0 0x7f8aa00cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.019+0000 7f8a9f7fe640 1 --2- 172.21.15.79:0/3717874901 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa015d130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009870 tx=0x7f8a9002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56885369c618ba00 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.019+0000 7f8a9e7fc640 1 -- 172.21.15.79:0/3717874901 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90037070 con 0x7f8aa0156670 2024-04-26T19:51:12.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.019+0000 7f8a9e7fc640 1 -- 172.21.15.79:0/3717874901 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a90032440 con 0x7f8aa0156670 2024-04-26T19:51:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.019+0000 7f8a9e7fc640 1 -- 172.21.15.79:0/3717874901 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90032810 con 0x7f8aa0156670 2024-04-26T19:51:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.020+0000 7f8aa5e93640 1 -- 172.21.15.79:0/3717874901 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 msgr2=0x7f8aa015d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.020+0000 7f8aa5e93640 1 --2- 172.21.15.79:0/3717874901 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa015d130 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009870 tx=0x7f8a9002a270 comp rx=0 tx=0).stop 2024-04-26T19:51:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.020+0000 7f8aa5e93640 1 -- 172.21.15.79:0/3717874901 shutdown_connections 2024-04-26T19:51:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.020+0000 7f8aa5e93640 1 --2- 172.21.15.79:0/3717874901 >> v2:172.21.15.83:3300/0 conn(0x7f8aa00b44f0 0x7f8aa00cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.020+0000 7f8aa5e93640 1 --2- 172.21.15.79:0/3717874901 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa015d130 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.020+0000 7f8aa5e93640 1 -- 172.21.15.79:0/3717874901 >> 172.21.15.79:0/3717874901 conn(0x7f8aa00b39d0 msgr2=0x7f8aa006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.020+0000 7f8aa5e93640 1 -- 172.21.15.79:0/3717874901 shutdown_connections 2024-04-26T19:51:12.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.020+0000 7f8aa5e93640 1 -- 172.21.15.79:0/3717874901 wait complete. 2024-04-26T19:51:12.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.021+0000 7f8aa5e93640 1 Processor -- start 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.021+0000 7f8aa5e93640 1 -- start start 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8aa5e93640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8aa00b44f0 0x7f8aa0142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8aa5e93640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aa00cd9c0 0x7f8aa0141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8aa5e93640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa0169580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8aa5e93640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0156a50 con 0x7f8aa00b44f0 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8aa5e93640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0072da0 con 0x7f8aa00cd9c0 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8aa5e93640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa0156bd0 con 0x7f8aa0156670 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8aa00b44f0 0x7f8aa0142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9f7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8aa00b44f0 0x7f8aa0142810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44154/0 (socket says 172.21.15.79:44154) 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9f7fe640 1 -- 172.21.15.79:0/823921608 learned_addr learned my addr 172.21.15.79:0/823921608 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:12.027 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9effd640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aa00cd9c0 0x7f8aa0141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9ffff640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa0169580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9f7fe640 1 -- 172.21.15.79:0/823921608 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 msgr2=0x7f8aa0169580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9f7fe640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa0169580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9f7fe640 1 -- 172.21.15.79:0/823921608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aa00cd9c0 msgr2=0x7f8aa0141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9f7fe640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aa00cd9c0 0x7f8aa0141ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.022+0000 7f8a9f7fe640 1 -- 172.21.15.79:0/823921608 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa0142420 con 0x7f8aa00b44f0 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.023+0000 7f8a9f7fe640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8aa00b44f0 0x7f8aa0142810 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a9002a2b0 tx=0x7f8a900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.023+0000 7f8a87fff640 1 -- 172.21.15.79:0/823921608 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90041030 con 0x7f8aa00b44f0 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.023+0000 7f8a87fff640 1 -- 172.21.15.79:0/823921608 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a90030440 con 0x7f8aa00b44f0 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.023+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa0157ff0 con 0x7f8aa00b44f0 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.023+0000 7f8a87fff640 1 -- 172.21.15.79:0/823921608 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90037040 con 0x7f8aa00b44f0 2024-04-26T19:51:12.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.023+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa01585b0 con 0x7f8aa00b44f0 2024-04-26T19:51:12.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.024+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa00506c0 con 0x7f8aa00b44f0 2024-04-26T19:51:12.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.024+0000 7f8a87fff640 1 -- 172.21.15.79:0/823921608 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a90044070 con 0x7f8aa00b44f0 2024-04-26T19:51:12.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.025+0000 7f8a87fff640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.025+0000 7f8a9effd640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.025+0000 7f8a87fff640 1 -- 172.21.15.79:0/823921608 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f8a900ac480 con 0x7f8aa00b44f0 2024-04-26T19:51:12.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.025+0000 7f8a9effd640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0155e50 tx=0x7f8a94007e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.029+0000 7f8a87fff640 1 -- 172.21.15.79:0/823921608 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f8aa0142420 con 0x7f8aa00b44f0 2024-04-26T19:51:12.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.205+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8aa00cfb10 con 0x7f8aa00b44f0 2024-04-26T19:51:12.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.206+0000 7f8a87fff640 1 -- 172.21.15.79:0/823921608 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18852 (secure 0 0 0) 0x7f8a90079f80 con 0x7f8aa00b44f0 2024-04-26T19:51:12.207 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:12.207 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":26,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:11.226379+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8a8c06c8f0 msgr2=0x7f8a8c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0155e50 tx=0x7f8a94007e80 comp rx=0 tx=0).stop 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8aa00b44f0 msgr2=0x7f8aa0142810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8aa00b44f0 0x7f8aa0142810 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8a9002a2b0 tx=0x7f8a900046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 shutdown_connections 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f8a8c06c8f0 0x7f8a8c06edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f8aa0156670 0x7f8aa0169580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f8aa00cd9c0 0x7f8aa0141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 --2- 172.21.15.79:0/823921608 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f8aa00b44f0 0x7f8aa0142810 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 >> 172.21.15.79:0/823921608 conn(0x7f8aa00b39d0 msgr2=0x7f8aa006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:12.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 shutdown_connections 2024-04-26T19:51:12.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.208+0000 7f8aa5e93640 1 -- 172.21.15.79:0/823921608 wait complete. 2024-04-26T19:51:12.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:12.221 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:12.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.324+0000 7fa1e617f640 1 Processor -- start 2024-04-26T19:51:12.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.324+0000 7fa1e617f640 1 -- start start 2024-04-26T19:51:12.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.325+0000 7fa1e617f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa1e00b3b90 0x7fa1e006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.325+0000 7fa1e617f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 0x7fa1e006c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.325+0000 7fa1e617f640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e0163370 con 0x7fa1e0162b30 2024-04-26T19:51:12.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.325+0000 7fa1e617f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e0162f10 con 0x7fa1e00b3b90 2024-04-26T19:51:12.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.325+0000 7fa1e617f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e0163120 con 0x7fa1e006d790 2024-04-26T19:51:12.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1df7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fa1e0162b30 0x7fa1e00b3480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:40002/0 (socket says 172.21.15.79:40002) 2024-04-26T19:51:12.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1df7fe640 1 -- 172.21.15.79:0/953068149 learned_addr learned my addr 172.21.15.79:0/953068149 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:12.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1deffd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa1e00b3b90 0x7fa1e006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1dffff640 1 --2- 172.21.15.79:0/953068149 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 0x7fa1e006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1de7fc640 1 -- 172.21.15.79:0/953068149 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1904363169 0 0) 0x7fa1e0163370 con 0x7fa1e0162b30 2024-04-26T19:51:12.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1de7fc640 1 -- 172.21.15.79:0/953068149 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1c8003520 con 0x7fa1e0162b30 2024-04-26T19:51:12.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1deffd640 1 -- 172.21.15.79:0/953068149 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 msgr2=0x7fa1e006c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:12.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1deffd640 1 --2- 172.21.15.79:0/953068149 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 0x7fa1e006c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1deffd640 1 -- 172.21.15.79:0/953068149 >> v1:172.21.15.83:6789/0 conn(0x7fa1e0162b30 legacy=0x7fa1e00b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1deffd640 1 -- 172.21.15.79:0/953068149 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e0163650 con 0x7fa1e00b3b90 2024-04-26T19:51:12.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.326+0000 7fa1dffff640 1 --2- 172.21.15.79:0/953068149 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 0x7fa1e006c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:12.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1deffd640 1 --2- 172.21.15.79:0/953068149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e00b3b90 0x7fa1e006c350 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40097a0 tx=0x7fa1d402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94663cb9b6a31f88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1de7fc640 1 -- 172.21.15.79:0/953068149 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4037070 con 0x7fa1e00b3b90 2024-04-26T19:51:12.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1de7fc640 1 -- 172.21.15.79:0/953068149 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d402ae50 con 0x7fa1e00b3b90 2024-04-26T19:51:12.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1de7fc640 1 -- 172.21.15.79:0/953068149 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4032640 con 0x7fa1e00b3b90 2024-04-26T19:51:12.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1e617f640 1 -- 172.21.15.79:0/953068149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e00b3b90 msgr2=0x7fa1e006c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1e617f640 1 --2- 172.21.15.79:0/953068149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e00b3b90 0x7fa1e006c350 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40097a0 tx=0x7fa1d402a100 comp rx=0 tx=0).stop 2024-04-26T19:51:12.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1e617f640 1 -- 172.21.15.79:0/953068149 shutdown_connections 2024-04-26T19:51:12.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1e617f640 1 --2- 172.21.15.79:0/953068149 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 0x7fa1e006c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1e617f640 1 --2- 172.21.15.79:0/953068149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e00b3b90 0x7fa1e006c350 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1e617f640 1 -- 172.21.15.79:0/953068149 >> 172.21.15.79:0/953068149 conn(0x7fa1e00b4500 msgr2=0x7fa1e00b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:12.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1e617f640 1 -- 172.21.15.79:0/953068149 shutdown_connections 2024-04-26T19:51:12.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.327+0000 7fa1e617f640 1 -- 172.21.15.79:0/953068149 wait complete. 2024-04-26T19:51:12.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.328+0000 7fa1e617f640 1 Processor -- start 2024-04-26T19:51:12.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.329+0000 7fa1e617f640 1 -- start start 2024-04-26T19:51:12.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.329+0000 7fa1e617f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 0x7fa1e0156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.329+0000 7fa1e617f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1e00b3b90 0x7fa1e006ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.329+0000 7fa1e617f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e0162b30 0x7fa1e006f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.329+0000 7fa1e617f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00d02f0 con 0x7fa1e00b3b90 2024-04-26T19:51:12.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.329+0000 7fa1e617f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e0163c10 con 0x7fa1e0162b30 2024-04-26T19:51:12.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.329+0000 7fa1e617f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e00b1900 con 0x7fa1e006d790 2024-04-26T19:51:12.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.329+0000 7fa1deffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1e00b3b90 0x7fa1e006ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1deffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1e00b3b90 0x7fa1e006ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44156/0 (socket says 172.21.15.79:44156) 2024-04-26T19:51:12.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1deffd640 1 -- 172.21.15.79:0/2717290530 learned_addr learned my addr 172.21.15.79:0/2717290530 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:12.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1dffff640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e0162b30 0x7fa1e006f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1df7fe640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 0x7fa1e0156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1deffd640 1 -- 172.21.15.79:0/2717290530 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 msgr2=0x7fa1e0156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1deffd640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 0x7fa1e0156da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1deffd640 1 -- 172.21.15.79:0/2717290530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e0162b30 msgr2=0x7fa1e006f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1deffd640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e0162b30 0x7fa1e006f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1deffd640 1 -- 172.21.15.79:0/2717290530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e00cea10 con 0x7fa1e00b3b90 2024-04-26T19:51:12.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1deffd640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1e00b3b90 0x7fa1e006ec80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4035040 tx=0x7fa1d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1c7fff640 1 -- 172.21.15.79:0/2717290530 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4037050 con 0x7fa1e00b3b90 2024-04-26T19:51:12.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1c7fff640 1 -- 172.21.15.79:0/2717290530 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d4038040 con 0x7fa1e00b3b90 2024-04-26T19:51:12.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e00cecf0 con 0x7fa1e00b3b90 2024-04-26T19:51:12.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1c7fff640 1 -- 172.21.15.79:0/2717290530 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4006e10 con 0x7fa1e00b3b90 2024-04-26T19:51:12.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.330+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e00cf2b0 con 0x7fa1e00b3b90 2024-04-26T19:51:12.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.331+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e00cf4f0 con 0x7fa1e00b3b90 2024-04-26T19:51:12.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.332+0000 7fa1c7fff640 1 -- 172.21.15.79:0/2717290530 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1d4045020 con 0x7fa1e00b3b90 2024-04-26T19:51:12.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.332+0000 7fa1c7fff640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1cc06c8f0 0x7fa1cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.332+0000 7fa1df7fe640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1cc06c8f0 0x7fa1cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.332+0000 7fa1c7fff640 1 -- 172.21.15.79:0/2717290530 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa1d40abf90 con 0x7fa1e00b3b90 2024-04-26T19:51:12.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.333+0000 7fa1df7fe640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1cc06c8f0 0x7fa1cc06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0005880 tx=0x7fa1d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.337+0000 7fa1c7fff640 1 -- 172.21.15.79:0/2717290530 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa1e00cf4f0 con 0x7fa1e00b3b90 2024-04-26T19:51:12.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.512+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa1e00506c0 con 0x7fa1e00b3b90 2024-04-26T19:51:12.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.512+0000 7fa1c7fff640 1 -- 172.21.15.79:0/2717290530 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fa1d4079fc0 con 0x7fa1e00b3b90 2024-04-26T19:51:12.513 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:12.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1cc06c8f0 msgr2=0x7fa1cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1cc06c8f0 0x7fa1cc06edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0005880 tx=0x7fa1d0009040 comp rx=0 tx=0).stop 2024-04-26T19:51:12.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1e00b3b90 msgr2=0x7fa1e006ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1e00b3b90 0x7fa1e006ec80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4035040 tx=0x7fa1d40046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 shutdown_connections 2024-04-26T19:51:12.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1cc06c8f0 0x7fa1cc06edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1e0162b30 0x7fa1e006f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1e00b3b90 0x7fa1e006ec80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 --2- 172.21.15.79:0/2717290530 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1e006d790 0x7fa1e0156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.514+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 >> 172.21.15.79:0/2717290530 conn(0x7fa1e00b4500 msgr2=0x7fa1e00d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:12.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.515+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 shutdown_connections 2024-04-26T19:51:12.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.515+0000 7fa1e617f640 1 -- 172.21.15.79:0/2717290530 wait complete. 2024-04-26T19:51:12.526 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:12.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.626+0000 7f77bc8d3640 1 Processor -- start 2024-04-26T19:51:12.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.626+0000 7f77bc8d3640 1 -- start start 2024-04-26T19:51:12.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.627+0000 7f77bc8d3640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f77b815c020 0x7f77b80aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.627+0000 7f77bc8d3640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b80aaf40 con 0x7f77b806f740 2024-04-26T19:51:12.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.627+0000 7f77bc8d3640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77b80ab220 con 0x7f77b815c400 2024-04-26T19:51:12.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.627+0000 7f77bc8d3640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f77b80ab500 con 0x7f77b815c020 2024-04-26T19:51:12.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.627+0000 7f77b659c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f77b815c020 0x7f77b80aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.627+0000 7f77b6d9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f77b806f740 0x7f77b80d0df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35198/0 (socket says 172.21.15.79:35198) 2024-04-26T19:51:12.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.628+0000 7f77b6d9d640 1 -- 172.21.15.79:0/3862249310 learned_addr learned my addr 172.21.15.79:0/3862249310 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:12.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.628+0000 7f77b659c640 1 -- 172.21.15.79:0/3862249310 >> v1:172.21.15.83:6789/0 conn(0x7f77b815c400 legacy=0x7f77b806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.628+0000 7f77b659c640 1 -- 172.21.15.79:0/3862249310 >> v1:172.21.15.79:6789/0 conn(0x7f77b806f740 legacy=0x7f77b80d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.628+0000 7f77b659c640 1 -- 172.21.15.79:0/3862249310 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b80d08d0 con 0x7f77b815c020 2024-04-26T19:51:12.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.628+0000 7f77b659c640 1 --2- 172.21.15.79:0/3862249310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815c020 0x7f77b80aa4c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009d90 tx=0x7f77a802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b7882799ba9211 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77b559a640 1 -- 172.21.15.79:0/3862249310 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8007740 con 0x7f77b815c020 2024-04-26T19:51:12.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77b559a640 1 -- 172.21.15.79:0/3862249310 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a80078e0 con 0x7f77b815c020 2024-04-26T19:51:12.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77b559a640 1 -- 172.21.15.79:0/3862249310 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8007cb0 con 0x7f77b815c020 2024-04-26T19:51:12.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77bc8d3640 1 -- 172.21.15.79:0/3862249310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815c020 msgr2=0x7f77b80aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77bc8d3640 1 --2- 172.21.15.79:0/3862249310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815c020 0x7f77b80aa4c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009d90 tx=0x7f77a802a5f0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77bc8d3640 1 -- 172.21.15.79:0/3862249310 shutdown_connections 2024-04-26T19:51:12.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77bc8d3640 1 --2- 172.21.15.79:0/3862249310 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815c020 0x7f77b80aa4c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77bc8d3640 1 -- 172.21.15.79:0/3862249310 >> 172.21.15.79:0/3862249310 conn(0x7f77b80b3ed0 msgr2=0x7f77b80681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:12.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77bc8d3640 1 -- 172.21.15.79:0/3862249310 shutdown_connections 2024-04-26T19:51:12.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.629+0000 7f77bc8d3640 1 -- 172.21.15.79:0/3862249310 wait complete. 2024-04-26T19:51:12.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77bc8d3640 1 Processor -- start 2024-04-26T19:51:12.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77bc8d3640 1 -- start start 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77bc8d3640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b806f740 0x7f77b81419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77bc8d3640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77b815c020 0x7f77b815a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77bc8d3640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815aa80 0x7f77b8071850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77bc8d3640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77b80aab80 con 0x7f77b815aa80 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77bc8d3640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77b80aaa00 con 0x7f77b806f740 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77bc8d3640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f77b80aad00 con 0x7f77b815c020 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77b6d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815aa80 0x7f77b8071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77b6d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815aa80 0x7f77b8071850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44164/0 (socket says 172.21.15.79:44164) 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77b6d9d640 1 -- 172.21.15.79:0/505200235 learned_addr learned my addr 172.21.15.79:0/505200235 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:12.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77b5d9b640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77b815c020 0x7f77b815a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.631+0000 7f77b659c640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b806f740 0x7f77b81419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77b6d9d640 1 -- 172.21.15.79:0/505200235 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77b815c020 msgr2=0x7f77b815a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77b6d9d640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77b815c020 0x7f77b815a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77b6d9d640 1 -- 172.21.15.79:0/505200235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b806f740 msgr2=0x7f77b81419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77b6d9d640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b806f740 0x7f77b81419c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77b6d9d640 1 -- 172.21.15.79:0/505200235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b81f6cd0 con 0x7f77b815aa80 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77b659c640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b806f740 0x7f77b81419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77b6d9d640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815aa80 0x7f77b8071850 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77b000c7b0 tx=0x7f77b000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77a6ffd640 1 -- 172.21.15.79:0/505200235 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0018070 con 0x7f77b815aa80 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77a6ffd640 1 -- 172.21.15.79:0/505200235 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b00072d0 con 0x7f77b815aa80 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b81f6fb0 con 0x7f77b815aa80 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77a6ffd640 1 -- 172.21.15.79:0/505200235 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b00047e0 con 0x7f77b815aa80 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.632+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b81f7570 con 0x7f77b815aa80 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.633+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77b80506c0 con 0x7f77b815aa80 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.633+0000 7f77a6ffd640 1 -- 172.21.15.79:0/505200235 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77b001b070 con 0x7f77b815aa80 2024-04-26T19:51:12.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.634+0000 7f77a6ffd640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f779806c8f0 0x7f779806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.634+0000 7f77a6ffd640 1 -- 172.21.15.79:0/505200235 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f77b008b290 con 0x7f77b815aa80 2024-04-26T19:51:12.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.634+0000 7f77b659c640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f779806c8f0 0x7f779806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.634+0000 7f77b659c640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f779806c8f0 0x7f779806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f77a8034000 tx=0x7f77a802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.638+0000 7f77a6ffd640 1 -- 172.21.15.79:0/505200235 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77b0058d00 con 0x7f77b815aa80 2024-04-26T19:51:12.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.816+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f77b80d0030 con 0x7f77b815aa80 2024-04-26T19:51:12.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.817+0000 7f77a6ffd640 1 -- 172.21.15.79:0/505200235 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f77b005d000 con 0x7f77b815aa80 2024-04-26T19:51:12.818 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:12.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f779806c8f0 msgr2=0x7f779806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f779806c8f0 0x7f779806edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f77a8034000 tx=0x7f77a802f000 comp rx=0 tx=0).stop 2024-04-26T19:51:12.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815aa80 msgr2=0x7f77b8071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815aa80 0x7f77b8071850 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f77b000c7b0 tx=0x7f77b000cc70 comp rx=0 tx=0).stop 2024-04-26T19:51:12.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 shutdown_connections 2024-04-26T19:51:12.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f779806c8f0 0x7f779806edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77b815aa80 0x7f77b8071850 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77b815c020 0x7f77b815a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 --2- 172.21.15.79:0/505200235 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77b806f740 0x7f77b81419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 >> 172.21.15.79:0/505200235 conn(0x7f77b80b3ed0 msgr2=0x7f77b8073510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:12.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 shutdown_connections 2024-04-26T19:51:12.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.819+0000 7f77bc8d3640 1 -- 172.21.15.79:0/505200235 wait complete. 2024-04-26T19:51:12.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:12.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.931+0000 7fbb2cf9e640 1 Processor -- start 2024-04-26T19:51:12.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.931+0000 7fbb2cf9e640 1 -- start start 2024-04-26T19:51:12.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb2cf9e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb28160640 0x7fbb280b99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb2cf9e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb28072f30 0x7fbb2806def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb2cf9e640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb280b9f00 con 0x7fbb2815f7d0 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb2cf9e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbb2806f270 con 0x7fbb28160640 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb2cf9e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbb2806f550 con 0x7fbb28072f30 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb26ffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbb2815f7d0 0x7fbb28072820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35214/0 (socket says 172.21.15.79:35214) 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb26ffd640 1 -- 172.21.15.79:0/3842243680 learned_addr learned my addr 172.21.15.79:0/3842243680 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb27fff640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb28072f30 0x7fbb2806def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb277fe640 1 --2- 172.21.15.79:0/3842243680 >> v2:172.21.15.83:3300/0 conn(0x7fbb28160640 0x7fbb280b99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3821061865 0 0) 0x7fbb280b9f00 con 0x7fbb2815f7d0 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.932+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbb10003520 con 0x7fbb2815f7d0 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 180100458 0 0) 0x7fbb10003520 con 0x7fbb2815f7d0 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb28072f30 msgr2=0x7fbb2806def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:12.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb28072f30 0x7fbb2806def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 >> v2:172.21.15.83:3300/0 conn(0x7fbb28160640 msgr2=0x7fbb280b99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 --2- 172.21.15.79:0/3842243680 >> v2:172.21.15.83:3300/0 conn(0x7fbb28160640 0x7fbb280b99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb2806f830 con 0x7fbb2815f7d0 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fbb1c001670 con 0x7fbb2815f7d0 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 >> v1:172.21.15.79:6789/0 conn(0x7fbb2815f7d0 legacy=0x7fbb28072820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb28072f30 0x7fbb100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb10003f30 0x7fbb10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb10004d40 0x7fbb100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2806f270 con 0x7fbb10004d40 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2806f550 con 0x7fbb10003f30 2024-04-26T19:51:12.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1000b9c0 con 0x7fbb28072f30 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb27fff640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb10004d40 0x7fbb100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb26ffd640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb10003f30 0x7fbb10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb277fe640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb28072f30 0x7fbb100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb27fff640 1 -- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb28072f30 msgr2=0x7fbb100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb27fff640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb28072f30 0x7fbb100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb27fff640 1 -- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb10003f30 msgr2=0x7fbb10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb27fff640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb10003f30 0x7fbb10004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.933+0000 7fbb27fff640 1 -- 172.21.15.79:0/3842243680 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb10003520 con 0x7fbb10004d40 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb27fff640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb10004d40 0x7fbb100051c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009f90 tx=0x7fbb10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0507cf5873977b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20009c10 con 0x7fbb10004d40 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb20036070 con 0x7fbb10004d40 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb267fc640 1 -- 172.21.15.79:0/3842243680 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20031670 con 0x7fbb10004d40 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/3842243680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb10004d40 msgr2=0x7fbb100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb2cf9e640 1 --2- 172.21.15.79:0/3842243680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb10004d40 0x7fbb100051c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009f90 tx=0x7fbb10005830 comp rx=0 tx=0).stop 2024-04-26T19:51:12.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb27fff640 1 -- 172.21.15.79:0/3842243680 reap_dead start 2024-04-26T19:51:12.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/3842243680 shutdown_connections 2024-04-26T19:51:12.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/3842243680 >> 172.21.15.79:0/3842243680 conn(0x7fbb280b4730 msgr2=0x7fbb280b4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:12.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/3842243680 shutdown_connections 2024-04-26T19:51:12.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.934+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/3842243680 wait complete. 2024-04-26T19:51:12.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.936+0000 7fbb2cf9e640 1 Processor -- start 2024-04-26T19:51:12.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.936+0000 7fbb2cf9e640 1 -- start start 2024-04-26T19:51:12.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb2cf9e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb10004d40 0x7fbb281f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb2cf9e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb28158720 0x7fbb280d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb2cf9e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb28148980 0x7fbb280d42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb2cf9e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbb280700d0 con 0x7fbb28148980 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb2cf9e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb2806fdf0 con 0x7fbb28158720 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb2cf9e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbb28160a20 con 0x7fbb10004d40 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb28148980 0x7fbb280d42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb27fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb28148980 0x7fbb280d42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44176/0 (socket says 172.21.15.79:44176) 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb27fff640 1 -- 172.21.15.79:0/4191411157 learned_addr learned my addr 172.21.15.79:0/4191411157 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb26ffd640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb28158720 0x7fbb280d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb27fff640 1 -- 172.21.15.79:0/4191411157 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb10004d40 msgr2=0x7fbb281f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb27fff640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb10004d40 0x7fbb281f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb27fff640 1 -- 172.21.15.79:0/4191411157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb28158720 msgr2=0x7fbb280d3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb27fff640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb28158720 0x7fbb280d3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb27fff640 1 -- 172.21.15.79:0/4191411157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb281f8390 con 0x7fbb28148980 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb27fff640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb28148980 0x7fbb280d42c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009f90 tx=0x7fbb20009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb0ffff640 1 -- 172.21.15.79:0/4191411157 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20009c10 con 0x7fbb28148980 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb0ffff640 1 -- 172.21.15.79:0/4191411157 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb20031d30 con 0x7fbb28148980 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb281f8670 con 0x7fbb28148980 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb0ffff640 1 -- 172.21.15.79:0/4191411157 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20003bb0 con 0x7fbb28148980 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.937+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb281f8c30 con 0x7fbb28148980 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.938+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb281f8e30 con 0x7fbb28148980 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.939+0000 7fbb0ffff640 1 -- 172.21.15.79:0/4191411157 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbb20005d10 con 0x7fbb28148980 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.939+0000 7fbb0ffff640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb1406c8f0 0x7fbb1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.939+0000 7fbb277fe640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb1406c8f0 0x7fbb1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.939+0000 7fbb0ffff640 1 -- 172.21.15.79:0/4191411157 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbb200acea0 con 0x7fbb28148980 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.940+0000 7fbb277fe640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb1406c8f0 0x7fbb1406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb18006e80 tx=0x7fbb18006df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:12.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:12.944+0000 7fbb0ffff640 1 -- 172.21.15.79:0/4191411157 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb281f8e30 con 0x7fbb28148980 2024-04-26T19:51:13.118 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.118+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbb28050f10 con 0x7fbb28148980 2024-04-26T19:51:13.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.118+0000 7fbb0ffff640 1 -- 172.21.15.79:0/4191411157 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fbb20049060 con 0x7fbb28148980 2024-04-26T19:51:13.119 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:13.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb1406c8f0 msgr2=0x7fbb1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb1406c8f0 0x7fbb1406edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbb18006e80 tx=0x7fbb18006df0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb28148980 msgr2=0x7fbb280d42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb28148980 0x7fbb280d42c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009f90 tx=0x7fbb20009750 comp rx=0 tx=0).stop 2024-04-26T19:51:13.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 shutdown_connections 2024-04-26T19:51:13.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb1406c8f0 0x7fbb1406edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb28148980 0x7fbb280d42c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb28158720 0x7fbb280d3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 --2- 172.21.15.79:0/4191411157 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb10004d40 0x7fbb281f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 >> 172.21.15.79:0/4191411157 conn(0x7fbb280b4730 msgr2=0x7fbb2815d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:13.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 shutdown_connections 2024-04-26T19:51:13.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.120+0000 7fbb2cf9e640 1 -- 172.21.15.79:0/4191411157 wait complete. 2024-04-26T19:51:13.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:13.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.231+0000 7fcb2eb40640 1 Processor -- start 2024-04-26T19:51:13.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.231+0000 7fcb2eb40640 1 -- start start 2024-04-26T19:51:13.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.231+0000 7fcb2eb40640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcb2815c1e0 0x7fcb28073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.231+0000 7fcb2eb40640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 0x7fcb2806fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2eb40640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb280718f0 0x7fcb28071d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2eb40640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fcb280702c0 con 0x7fcb280718f0 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2eb40640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcb28072620 con 0x7fcb2815c1e0 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2eb40640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fcb280730d0 con 0x7fcb2806f920 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2d0dd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb280718f0 0x7fcb28071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2d0dd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fcb280718f0 0x7fcb28071d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44192/0 (socket says 172.21.15.79:44192) 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2d0dd640 1 -- 172.21.15.79:0/3556963866 learned_addr learned my addr 172.21.15.79:0/3556963866 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2c8dc640 1 --2- 172.21.15.79:0/3556963866 >> v2:172.21.15.83:3300/0 conn(0x7fcb2815c1e0 0x7fcb28073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2d0dd640 1 -- 172.21.15.79:0/3556963866 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 msgr2=0x7fcb2806fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2d0dd640 1 --2- 172.21.15.79:0/3556963866 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 0x7fcb2806fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2d0dd640 1 -- 172.21.15.79:0/3556963866 >> v2:172.21.15.83:3300/0 conn(0x7fcb2815c1e0 msgr2=0x7fcb28073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:13.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2d0dd640 1 --2- 172.21.15.79:0/3556963866 >> v2:172.21.15.83:3300/0 conn(0x7fcb2815c1e0 0x7fcb28073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2d0dd640 1 -- 172.21.15.79:0/3556963866 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb280733b0 con 0x7fcb280718f0 2024-04-26T19:51:13.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.232+0000 7fcb2c8dc640 1 --2- 172.21.15.79:0/3556963866 >> v2:172.21.15.83:3300/0 conn(0x7fcb2815c1e0 0x7fcb28073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:13.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2d0dd640 1 --2- 172.21.15.79:0/3556963866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 0x7fcb28071d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcb280d1160 tx=0x7fcb2002a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edef50be3d950f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:13.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb277fe640 1 -- 172.21.15.79:0/3556963866 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20036070 con 0x7fcb280718f0 2024-04-26T19:51:13.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb277fe640 1 -- 172.21.15.79:0/3556963866 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb20031600 con 0x7fcb280718f0 2024-04-26T19:51:13.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb277fe640 1 -- 172.21.15.79:0/3556963866 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb200319d0 con 0x7fcb280718f0 2024-04-26T19:51:13.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2eb40640 1 -- 172.21.15.79:0/3556963866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 msgr2=0x7fcb28071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/3556963866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 0x7fcb28071d40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcb280d1160 tx=0x7fcb2002a570 comp rx=0 tx=0).stop 2024-04-26T19:51:13.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2eb40640 1 -- 172.21.15.79:0/3556963866 shutdown_connections 2024-04-26T19:51:13.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/3556963866 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 0x7fcb28071d40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/3556963866 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 0x7fcb2806fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/3556963866 >> v2:172.21.15.83:3300/0 conn(0x7fcb2815c1e0 0x7fcb28073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2eb40640 1 -- 172.21.15.79:0/3556963866 >> 172.21.15.79:0/3556963866 conn(0x7fcb280b2540 msgr2=0x7fcb280b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:13.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2eb40640 1 -- 172.21.15.79:0/3556963866 shutdown_connections 2024-04-26T19:51:13.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.233+0000 7fcb2eb40640 1 -- 172.21.15.79:0/3556963866 wait complete. 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.234+0000 7fcb2eb40640 1 Processor -- start 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.234+0000 7fcb2eb40640 1 -- start start 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.235+0000 7fcb2eb40640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 0x7fcb2814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.235+0000 7fcb2eb40640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 0x7fcb280b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.235+0000 7fcb2eb40640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2815c1e0 0x7fcb280b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.235+0000 7fcb2eb40640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcb28072400 con 0x7fcb280718f0 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.235+0000 7fcb2eb40640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcb28072280 con 0x7fcb2815c1e0 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.235+0000 7fcb2eb40640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fcb280cdc90 con 0x7fcb2806f920 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.235+0000 7fcb27fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 0x7fcb280b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.235+0000 7fcb27fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 0x7fcb280b07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44194/0 (socket says 172.21.15.79:44194) 2024-04-26T19:51:13.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.235+0000 7fcb27fff640 1 -- 172.21.15.79:0/1440835176 learned_addr learned my addr 172.21.15.79:0/1440835176 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb2c8dc640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 0x7fcb2814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb2d0dd640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2815c1e0 0x7fcb280b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb27fff640 1 -- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 msgr2=0x7fcb2814e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb27fff640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 0x7fcb2814e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb27fff640 1 -- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2815c1e0 msgr2=0x7fcb280b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb27fff640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2815c1e0 0x7fcb280b0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb27fff640 1 -- 172.21.15.79:0/1440835176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb280b1470 con 0x7fcb280718f0 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb2c8dc640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 0x7fcb2814e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb2d0dd640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2815c1e0 0x7fcb280b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb27fff640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 0x7fcb280b07f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400b4a0 tx=0x7fcb1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb257fa640 1 -- 172.21.15.79:0/1440835176 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14014070 con 0x7fcb280718f0 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb280b1750 con 0x7fcb280718f0 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.236+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb280b1d10 con 0x7fcb280718f0 2024-04-26T19:51:13.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.237+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb280506c0 con 0x7fcb280718f0 2024-04-26T19:51:13.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.238+0000 7fcb257fa640 1 -- 172.21.15.79:0/1440835176 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb14010610 con 0x7fcb280718f0 2024-04-26T19:51:13.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.238+0000 7fcb257fa640 1 -- 172.21.15.79:0/1440835176 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1401e060 con 0x7fcb280718f0 2024-04-26T19:51:13.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.239+0000 7fcb257fa640 1 -- 172.21.15.79:0/1440835176 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcb14017070 con 0x7fcb280718f0 2024-04-26T19:51:13.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.239+0000 7fcb257fa640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.240+0000 7fcb257fa640 1 -- 172.21.15.79:0/1440835176 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcb1408bef0 con 0x7fcb280718f0 2024-04-26T19:51:13.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.240+0000 7fcb2c8dc640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.240+0000 7fcb2c8dc640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00a990 tx=0x7fcb1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:13.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.243+0000 7fcb257fa640 1 -- 172.21.15.79:0/1440835176 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcb14059f20 con 0x7fcb280718f0 2024-04-26T19:51:13.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.419+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fcb280cda70 con 0x7fcb280718f0 2024-04-26T19:51:13.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.420+0000 7fcb257fa640 1 -- 172.21.15.79:0/1440835176 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fcb14059820 con 0x7fcb280718f0 2024-04-26T19:51:13.420 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:13.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.421+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcb0c06c8f0 msgr2=0x7fcb0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.421+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00a990 tx=0x7fcb1c008040 comp rx=0 tx=0).stop 2024-04-26T19:51:13.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.421+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 msgr2=0x7fcb280b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.421+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 0x7fcb280b07f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400b4a0 tx=0x7fcb1400b960 comp rx=0 tx=0).stop 2024-04-26T19:51:13.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.421+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 shutdown_connections 2024-04-26T19:51:13.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.421+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fcb0c06c8f0 0x7fcb0c06edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.422+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcb2815c1e0 0x7fcb280b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.422+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcb280718f0 0x7fcb280b07f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.422+0000 7fcb2eb40640 1 --2- 172.21.15.79:0/1440835176 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcb2806f920 0x7fcb2814e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.422+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 >> 172.21.15.79:0/1440835176 conn(0x7fcb280b2540 msgr2=0x7fcb280b2920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:13.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.422+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 shutdown_connections 2024-04-26T19:51:13.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.422+0000 7fcb2eb40640 1 -- 172.21.15.79:0/1440835176 wait complete. 2024-04-26T19:51:13.434 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:13.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.530+0000 7f0d195c6640 1 Processor -- start 2024-04-26T19:51:13.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.530+0000 7f0d195c6640 1 -- start start 2024-04-26T19:51:13.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d195c6640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d1406fb20 0x7f0d140d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d195c6640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0d1406ff00 0x7f0d140d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d195c6640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d140b1290 con 0x7f0d141488c0 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d195c6640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f0d140d3e60 con 0x7f0d1406ff00 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d195c6640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0d140d4090 con 0x7f0d1406fb20 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d127fc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0d1406ff00 0x7f0d140d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d127fc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f0d1406ff00 0x7f0d140d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44200/0 (socket says 172.21.15.79:44200) 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d127fc640 1 -- 172.21.15.79:0/1534762818 learned_addr learned my addr 172.21.15.79:0/1534762818 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d12ffd640 1 --2- 172.21.15.79:0/1534762818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d1406fb20 0x7f0d140d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d127fc640 1 -- 172.21.15.79:0/1534762818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d1406fb20 msgr2=0x7f0d140d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d127fc640 1 --2- 172.21.15.79:0/1534762818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d1406fb20 0x7f0d140d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d127fc640 1 -- 172.21.15.79:0/1534762818 >> v1:172.21.15.83:6789/0 conn(0x7f0d141488c0 legacy=0x7f0d14148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d127fc640 1 -- 172.21.15.79:0/1534762818 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d140d4370 con 0x7f0d1406ff00 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.531+0000 7f0d12ffd640 1 --2- 172.21.15.79:0/1534762818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d1406fb20 0x7f0d140d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.532+0000 7f0d127fc640 1 --2- 172.21.15.79:0/1534762818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 0x7f0d140d3920 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009870 tx=0x7f0cfc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b03e4c22d49164 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:13.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.532+0000 7f0d11ffb640 1 -- 172.21.15.79:0/1534762818 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc037070 con 0x7f0d1406ff00 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.532+0000 7f0d11ffb640 1 -- 172.21.15.79:0/1534762818 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cfc02ae50 con 0x7f0d1406ff00 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.532+0000 7f0d11ffb640 1 -- 172.21.15.79:0/1534762818 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc032860 con 0x7f0d1406ff00 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.533+0000 7f0d195c6640 1 -- 172.21.15.79:0/1534762818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 msgr2=0x7f0d140d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.533+0000 7f0d195c6640 1 --2- 172.21.15.79:0/1534762818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 0x7f0d140d3920 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009870 tx=0x7f0cfc02a300 comp rx=0 tx=0).stop 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.533+0000 7f0d195c6640 1 -- 172.21.15.79:0/1534762818 shutdown_connections 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.533+0000 7f0d195c6640 1 --2- 172.21.15.79:0/1534762818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 0x7f0d140d3920 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.533+0000 7f0d195c6640 1 --2- 172.21.15.79:0/1534762818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d1406fb20 0x7f0d140d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.533+0000 7f0d195c6640 1 -- 172.21.15.79:0/1534762818 >> 172.21.15.79:0/1534762818 conn(0x7f0d14071c30 msgr2=0x7f0d1406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.533+0000 7f0d195c6640 1 -- 172.21.15.79:0/1534762818 shutdown_connections 2024-04-26T19:51:13.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.533+0000 7f0d195c6640 1 -- 172.21.15.79:0/1534762818 wait complete. 2024-04-26T19:51:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.534+0000 7f0d195c6640 1 Processor -- start 2024-04-26T19:51:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.534+0000 7f0d195c6640 1 -- start start 2024-04-26T19:51:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d195c6640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1406fb20 0x7f0d14161590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d195c6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 0x7f0d14138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d195c6640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d141488c0 0x7f0d140b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d195c6640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0d140d4c10 con 0x7f0d1406ff00 2024-04-26T19:51:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d195c6640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0d140d4930 con 0x7f0d1406fb20 2024-04-26T19:51:13.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d195c6640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0d140aebb0 con 0x7f0d141488c0 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d127fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 0x7f0d14138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d127fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 0x7f0d14138ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44208/0 (socket says 172.21.15.79:44208) 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d127fc640 1 -- 172.21.15.79:0/1787225704 learned_addr learned my addr 172.21.15.79:0/1787225704 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d137fe640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d141488c0 0x7f0d140b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d12ffd640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1406fb20 0x7f0d14161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d127fc640 1 -- 172.21.15.79:0/1787225704 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d141488c0 msgr2=0x7f0d140b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d127fc640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d141488c0 0x7f0d140b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d127fc640 1 -- 172.21.15.79:0/1787225704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1406fb20 msgr2=0x7f0d14161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d127fc640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1406fb20 0x7f0d14161590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d127fc640 1 -- 172.21.15.79:0/1787225704 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d14161d30 con 0x7f0d1406ff00 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d127fc640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 0x7f0d14138ca0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02a340 tx=0x7f0cfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:13.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d0b7fe640 1 -- 172.21.15.79:0/1787225704 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc041020 con 0x7f0d1406ff00 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d0b7fe640 1 -- 172.21.15.79:0/1787225704 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cfc03c670 con 0x7f0d1406ff00 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d14162010 con 0x7f0d1406ff00 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d0b7fe640 1 -- 172.21.15.79:0/1787225704 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc037040 con 0x7f0d1406ff00 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.535+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d141625d0 con 0x7f0d1406ff00 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.536+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d140506c0 con 0x7f0d1406ff00 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.537+0000 7f0d0b7fe640 1 -- 172.21.15.79:0/1787225704 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0cfc044020 con 0x7f0d1406ff00 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.537+0000 7f0d0b7fe640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf406c8f0 0x7f0cf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.537+0000 7f0d0b7fe640 1 -- 172.21.15.79:0/1787225704 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0cfc0ac820 con 0x7f0d1406ff00 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.537+0000 7f0d12ffd640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf406c8f0 0x7f0cf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.538+0000 7f0d12ffd640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf406c8f0 0x7f0cf406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009f70 tx=0x7f0d0400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:13.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.542+0000 7f0d0b7fe640 1 -- 172.21.15.79:0/1787225704 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0cfc04b030 con 0x7f0d1406ff00 2024-04-26T19:51:13.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.719+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d140d3700 con 0x7f0d1406ff00 2024-04-26T19:51:13.721 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.720+0000 7f0d0b7fe640 1 -- 172.21.15.79:0/1787225704 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7f0cfc07a720 con 0x7f0d1406ff00 2024-04-26T19:51:13.721 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:13.721 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:13.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf406c8f0 msgr2=0x7f0cf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf406c8f0 0x7f0cf406edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009f70 tx=0x7f0d0400b040 comp rx=0 tx=0).stop 2024-04-26T19:51:13.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 msgr2=0x7f0d14138ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 0x7f0d14138ca0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02a340 tx=0x7f0cfc0046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 shutdown_connections 2024-04-26T19:51:13.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f0cf406c8f0 0x7f0cf406edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0d141488c0 0x7f0d140b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0d1406ff00 0x7f0d14138ca0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 --2- 172.21.15.79:0/1787225704 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0d1406fb20 0x7f0d14161590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 >> 172.21.15.79:0/1787225704 conn(0x7f0d14071c30 msgr2=0x7f0d14147940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:13.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 shutdown_connections 2024-04-26T19:51:13.724 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.722+0000 7f0d195c6640 1 -- 172.21.15.79:0/1787225704 wait complete. 2024-04-26T19:51:13.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:13.735 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.833+0000 7fdd84ada640 1 Processor -- start 2024-04-26T19:51:13.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.833+0000 7fdd84ada640 1 -- start start 2024-04-26T19:51:13.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd84ada640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fdd800b9090 0x7fdd8015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd84ada640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd8006ca60 0x7fdd801610e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd84ada640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd800b9470 con 0x7fdd800d1850 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd84ada640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fdd800d1cc0 con 0x7fdd8006ca60 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd84ada640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fdd80160c50 con 0x7fdd800b9090 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd7ed9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd8006ca60 0x7fdd801610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd7ed9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdd8006ca60 0x7fdd801610e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44224/0 (socket says 172.21.15.79:44224) 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd7ed9d640 1 -- 172.21.15.79:0/3360036273 learned_addr learned my addr 172.21.15.79:0/3360036273 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd7e59c640 1 --2- 172.21.15.79:0/3360036273 >> v2:172.21.15.83:3300/0 conn(0x7fdd800b9090 0x7fdd8015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd7ed9d640 1 -- 172.21.15.79:0/3360036273 >> v2:172.21.15.83:3300/0 conn(0x7fdd800b9090 msgr2=0x7fdd8015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd7ed9d640 1 --2- 172.21.15.79:0/3360036273 >> v2:172.21.15.83:3300/0 conn(0x7fdd800b9090 0x7fdd8015fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd7ed9d640 1 -- 172.21.15.79:0/3360036273 >> v1:172.21.15.83:6789/0 conn(0x7fdd800d1850 legacy=0x7fdd8006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.834+0000 7fdd7ed9d640 1 -- 172.21.15.79:0/3360036273 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd800d1ee0 con 0x7fdd8006ca60 2024-04-26T19:51:13.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.835+0000 7fdd7ed9d640 1 --2- 172.21.15.79:0/3360036273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd8006ca60 0x7fdd801610e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd800d0ad0 tx=0x7fdd7802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d024adaf6f7e6939 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.835+0000 7fdd7d59a640 1 -- 172.21.15.79:0/3360036273 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78037070 con 0x7fdd8006ca60 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.835+0000 7fdd7d59a640 1 -- 172.21.15.79:0/3360036273 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd78032440 con 0x7fdd8006ca60 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.835+0000 7fdd7d59a640 1 -- 172.21.15.79:0/3360036273 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78032810 con 0x7fdd8006ca60 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.836+0000 7fdd84ada640 1 -- 172.21.15.79:0/3360036273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd8006ca60 msgr2=0x7fdd801610e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.836+0000 7fdd84ada640 1 --2- 172.21.15.79:0/3360036273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd8006ca60 0x7fdd801610e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdd800d0ad0 tx=0x7fdd7802a2c0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.836+0000 7fdd84ada640 1 -- 172.21.15.79:0/3360036273 shutdown_connections 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.836+0000 7fdd84ada640 1 --2- 172.21.15.79:0/3360036273 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd8006ca60 0x7fdd801610e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.836+0000 7fdd84ada640 1 --2- 172.21.15.79:0/3360036273 >> v2:172.21.15.83:3300/0 conn(0x7fdd800b9090 0x7fdd8015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.836+0000 7fdd84ada640 1 -- 172.21.15.79:0/3360036273 >> 172.21.15.79:0/3360036273 conn(0x7fdd80073b40 msgr2=0x7fdd80073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.836+0000 7fdd84ada640 1 -- 172.21.15.79:0/3360036273 shutdown_connections 2024-04-26T19:51:13.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.836+0000 7fdd84ada640 1 -- 172.21.15.79:0/3360036273 wait complete. 2024-04-26T19:51:13.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.837+0000 7fdd84ada640 1 Processor -- start 2024-04-26T19:51:13.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.837+0000 7fdd84ada640 1 -- start start 2024-04-26T19:51:13.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd84ada640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd8006ca60 0x7fdd8014ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd84ada640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800b9090 0x7fdd80152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd84ada640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd800d1850 0x7fdd8006f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd84ada640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdd800d2780 con 0x7fdd800d1850 2024-04-26T19:51:13.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd84ada640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdd800d24a0 con 0x7fdd800b9090 2024-04-26T19:51:13.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd84ada640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8006d030 con 0x7fdd8006ca60 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7ed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd800d1850 0x7fdd8006f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7ed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd800d1850 0x7fdd8006f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44240/0 (socket says 172.21.15.79:44240) 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7ed9d640 1 -- 172.21.15.79:0/2819825984 learned_addr learned my addr 172.21.15.79:0/2819825984 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7e59c640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd8006ca60 0x7fdd8014ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7ed9d640 1 -- 172.21.15.79:0/2819825984 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd8006ca60 msgr2=0x7fdd8014ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7dd9b640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800b9090 0x7fdd80152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7ed9d640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd8006ca60 0x7fdd8014ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7ed9d640 1 -- 172.21.15.79:0/2819825984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800b9090 msgr2=0x7fdd80152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7ed9d640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800b9090 0x7fdd80152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.838+0000 7fdd7ed9d640 1 -- 172.21.15.79:0/2819825984 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8014cf80 con 0x7fdd800d1850 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.839+0000 7fdd7ed9d640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd800d1850 0x7fdd8006f640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd780090b0 tx=0x7fdd780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.839+0000 7fdd6effd640 1 -- 172.21.15.79:0/2819825984 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78041020 con 0x7fdd800d1850 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.839+0000 7fdd6effd640 1 -- 172.21.15.79:0/2819825984 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd780306c0 con 0x7fdd800d1850 2024-04-26T19:51:13.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.839+0000 7fdd6effd640 1 -- 172.21.15.79:0/2819825984 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78037040 con 0x7fdd800d1850 2024-04-26T19:51:13.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.839+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd801f8ea0 con 0x7fdd800d1850 2024-04-26T19:51:13.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.839+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd801f9460 con 0x7fdd800d1850 2024-04-26T19:51:13.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.840+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd800506c0 con 0x7fdd800d1850 2024-04-26T19:51:13.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.840+0000 7fdd6effd640 1 -- 172.21.15.79:0/2819825984 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd78044020 con 0x7fdd800d1850 2024-04-26T19:51:13.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.840+0000 7fdd6effd640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c920 0x7fdd6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:13.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.840+0000 7fdd6effd640 1 -- 172.21.15.79:0/2819825984 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdd780ac680 con 0x7fdd800d1850 2024-04-26T19:51:13.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.840+0000 7fdd7e59c640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c920 0x7fdd6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:13.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.841+0000 7fdd7e59c640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c920 0x7fdd6006ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009f30 tx=0x7fdd7000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:13.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:13.845+0000 7fdd6effd640 1 -- 172.21.15.79:0/2819825984 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdd7804b030 con 0x7fdd800d1850 2024-04-26T19:51:14.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.017+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdd8006c3f0 con 0x7fdd800d1850 2024-04-26T19:51:14.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.017+0000 7fdd6effd640 1 -- 172.21.15.79:0/2819825984 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fdd7807a740 con 0x7fdd800d1850 2024-04-26T19:51:14.018 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:14.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c920 msgr2=0x7fdd6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c920 0x7fdd6006ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009f30 tx=0x7fdd7000b040 comp rx=0 tx=0).stop 2024-04-26T19:51:14.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd800d1850 msgr2=0x7fdd8006f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd800d1850 0x7fdd8006f640 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdd780090b0 tx=0x7fdd780046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 shutdown_connections 2024-04-26T19:51:14.020 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fdd6006c920 0x7fdd6006ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdd800d1850 0x7fdd8006f640 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdd800b9090 0x7fdd80152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 --2- 172.21.15.79:0/2819825984 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdd8006ca60 0x7fdd8014ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 >> 172.21.15.79:0/2819825984 conn(0x7fdd80073b40 msgr2=0x7fdd800b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:14.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 shutdown_connections 2024-04-26T19:51:14.021 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.019+0000 7fdd84ada640 1 -- 172.21.15.79:0/2819825984 wait complete. 2024-04-26T19:51:14.031 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:14.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.130+0000 7f467442c640 1 Processor -- start 2024-04-26T19:51:14.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.130+0000 7f467442c640 1 -- start start 2024-04-26T19:51:14.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.131+0000 7f467442c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f466c15c020 0x7f466c0aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.131+0000 7f467442c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 0x7f466c0d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.131+0000 7f467442c640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f466c0aaf40 con 0x7f466c15c400 2024-04-26T19:51:14.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.131+0000 7f467442c640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f466c0ab220 con 0x7f466c15c020 2024-04-26T19:51:14.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.131+0000 7f467442c640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f466c0ab500 con 0x7f466c06f740 2024-04-26T19:51:14.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.131+0000 7f46721c8640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f466c15c020 0x7f466c0aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.131+0000 7f46719c7640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f466c15c400 0x7f466c06f030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35220/0 (socket says 172.21.15.79:35220) 2024-04-26T19:51:14.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.131+0000 7f46719c7640 1 -- 172.21.15.79:0/1828685120 learned_addr learned my addr 172.21.15.79:0/1828685120 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:14.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.131+0000 7f46729c9640 1 --2- 172.21.15.79:0/1828685120 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 0x7f466c0d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.132+0000 7f46721c8640 1 -- 172.21.15.79:0/1828685120 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 msgr2=0x7f466c0d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:14.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.132+0000 7f46721c8640 1 --2- 172.21.15.79:0/1828685120 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 0x7f466c0d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.132+0000 7f46711c6640 1 -- 172.21.15.79:0/1828685120 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2925473996 0 0) 0x7f466c0aaf40 con 0x7f466c15c400 2024-04-26T19:51:14.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.132+0000 7f46721c8640 1 -- 172.21.15.79:0/1828685120 >> v1:172.21.15.79:6789/0 conn(0x7f466c15c400 legacy=0x7f466c06f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.132+0000 7f46721c8640 1 -- 172.21.15.79:0/1828685120 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c0d08d0 con 0x7f466c15c020 2024-04-26T19:51:14.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.132+0000 7f46721c8640 1 --2- 172.21.15.79:0/1828685120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 0x7f466c0aa4c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4668009d90 tx=0x7f466802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269cd3ff69d1b143 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.132+0000 7f46711c6640 1 -- 172.21.15.79:0/1828685120 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668007740 con 0x7f466c15c020 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f46711c6640 1 -- 172.21.15.79:0/1828685120 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46680078e0 con 0x7f466c15c020 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f46711c6640 1 -- 172.21.15.79:0/1828685120 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668007cb0 con 0x7f466c15c020 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f467442c640 1 -- 172.21.15.79:0/1828685120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 msgr2=0x7f466c0aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f467442c640 1 --2- 172.21.15.79:0/1828685120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 0x7f466c0aa4c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4668009d90 tx=0x7f466802a660 comp rx=0 tx=0).stop 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f467442c640 1 -- 172.21.15.79:0/1828685120 shutdown_connections 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f467442c640 1 --2- 172.21.15.79:0/1828685120 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 0x7f466c0d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f467442c640 1 --2- 172.21.15.79:0/1828685120 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 0x7f466c0aa4c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f467442c640 1 -- 172.21.15.79:0/1828685120 >> 172.21.15.79:0/1828685120 conn(0x7f466c0b3ed0 msgr2=0x7f466c0681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f467442c640 1 -- 172.21.15.79:0/1828685120 shutdown_connections 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.133+0000 7f467442c640 1 -- 172.21.15.79:0/1828685120 wait complete. 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.134+0000 7f467442c640 1 Processor -- start 2024-04-26T19:51:14.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.134+0000 7f467442c640 1 -- start start 2024-04-26T19:51:14.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f467442c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 0x7f466c13c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f467442c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 0x7f466c169370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f467442c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f466c1698b0 0x7f466c151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f467442c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f466c0aab80 con 0x7f466c15c020 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f467442c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f466c0aaa00 con 0x7f466c1698b0 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f467442c640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f466c0aad00 con 0x7f466c06f740 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46719c7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 0x7f466c169370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46719c7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 0x7f466c169370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44252/0 (socket says 172.21.15.79:44252) 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46719c7640 1 -- 172.21.15.79:0/1674290688 learned_addr learned my addr 172.21.15.79:0/1674290688 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46729c9640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f466c1698b0 0x7f466c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46721c8640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 0x7f466c13c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46719c7640 1 -- 172.21.15.79:0/1674290688 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 msgr2=0x7f466c13c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46719c7640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 0x7f466c13c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46719c7640 1 -- 172.21.15.79:0/1674290688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f466c1698b0 msgr2=0x7f466c151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46719c7640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f466c1698b0 0x7f466c151460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46719c7640 1 -- 172.21.15.79:0/1674290688 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c13cb90 con 0x7f466c15c020 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46729c9640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f466c1698b0 0x7f466c151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.135+0000 7f46719c7640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 0x7f466c169370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c460 tx=0x7f465c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.136+0000 7f4662ffd640 1 -- 172.21.15.79:0/1674290688 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c018070 con 0x7f466c15c020 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.136+0000 7f4662ffd640 1 -- 172.21.15.79:0/1674290688 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f465c007500 con 0x7f466c15c020 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.136+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c073db0 con 0x7f466c15c020 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.136+0000 7f4662ffd640 1 -- 172.21.15.79:0/1674290688 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c004a20 con 0x7f466c15c020 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.136+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c074370 con 0x7f466c15c020 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.137+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466c0506c0 con 0x7f466c15c020 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.137+0000 7f4662ffd640 1 -- 172.21.15.79:0/1674290688 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f465c01b070 con 0x7f466c15c020 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.137+0000 7f4662ffd640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f465006c8f0 0x7f465006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.137+0000 7f4662ffd640 1 -- 172.21.15.79:0/1674290688 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f465c08b100 con 0x7f466c15c020 2024-04-26T19:51:14.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.137+0000 7f46721c8640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f465006c8f0 0x7f465006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.138+0000 7f46721c8640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f465006c8f0 0x7f465006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4668034000 tx=0x7f466802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:14.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.142+0000 7f4662ffd640 1 -- 172.21.15.79:0/1674290688 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f465c0591c0 con 0x7f466c15c020 2024-04-26T19:51:14.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.317+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f466c14c610 con 0x7f466c15c020 2024-04-26T19:51:14.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.317+0000 7f4662ffd640 1 -- 172.21.15.79:0/1674290688 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f465c058ac0 con 0x7f466c15c020 2024-04-26T19:51:14.318 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:14.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f465006c8f0 msgr2=0x7f465006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f465006c8f0 0x7f465006edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4668034000 tx=0x7f466802f000 comp rx=0 tx=0).stop 2024-04-26T19:51:14.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 msgr2=0x7f466c169370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 0x7f466c169370 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f465c00c460 tx=0x7f465c00c920 comp rx=0 tx=0).stop 2024-04-26T19:51:14.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 shutdown_connections 2024-04-26T19:51:14.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f465006c8f0 0x7f465006edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f466c1698b0 0x7f466c151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f466c15c020 0x7f466c169370 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 --2- 172.21.15.79:0/1674290688 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f466c06f740 0x7f466c13c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.319+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 >> 172.21.15.79:0/1674290688 conn(0x7f466c0b3ed0 msgr2=0x7f466c15c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:14.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.320+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 shutdown_connections 2024-04-26T19:51:14.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.320+0000 7f467442c640 1 -- 172.21.15.79:0/1674290688 wait complete. 2024-04-26T19:51:14.331 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:14.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.429+0000 7fa358a2a640 1 Processor -- start 2024-04-26T19:51:14.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.429+0000 7fa358a2a640 1 -- start start 2024-04-26T19:51:14.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.430+0000 7fa358a2a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa354070030 0x7fa354073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.430+0000 7fa358a2a640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa354072620 con 0x7fa3540718f0 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.430+0000 7fa358a2a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3540730d0 con 0x7fa35415c1e0 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.430+0000 7fa358a2a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa3540733b0 con 0x7fa354070030 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.430+0000 7fa352d9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa3540718f0 0x7fa354071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35236/0 (socket says 172.21.15.79:35236) 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.430+0000 7fa352d9d640 1 -- 172.21.15.79:0/2824612530 learned_addr learned my addr 172.21.15.79:0/2824612530 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.431+0000 7fa34bfff640 1 --2- 172.21.15.79:0/2824612530 >> v2:172.21.15.83:3300/0 conn(0x7fa354070030 0x7fa354073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.431+0000 7fa351d9b640 1 -- 172.21.15.79:0/2824612530 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 364202627 0 0) 0x7fa354072620 con 0x7fa3540718f0 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.431+0000 7fa351d9b640 1 -- 172.21.15.79:0/2824612530 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa340003520 con 0x7fa3540718f0 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.431+0000 7fa34bfff640 1 -- 172.21.15.79:0/2824612530 >> v1:172.21.15.83:6789/0 conn(0x7fa35415c1e0 legacy=0x7fa35406f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.431+0000 7fa34bfff640 1 -- 172.21.15.79:0/2824612530 >> v1:172.21.15.79:6789/0 conn(0x7fa3540718f0 legacy=0x7fa354071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.431+0000 7fa351d9b640 1 -- 172.21.15.79:0/2824612530 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 151061355 0 0) 0x7fa340003520 con 0x7fa3540718f0 2024-04-26T19:51:14.432 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.431+0000 7fa34bfff640 1 -- 172.21.15.79:0/2824612530 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa354073690 con 0x7fa354070030 2024-04-26T19:51:14.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa34bfff640 1 --2- 172.21.15.79:0/2824612530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa354070030 0x7fa354073bb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0097a0 tx=0x7fa33c02a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25754c0359ce2b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa351d9b640 1 -- 172.21.15.79:0/2824612530 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c037070 con 0x7fa354070030 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa351d9b640 1 -- 172.21.15.79:0/2824612530 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa33c02ab30 con 0x7fa354070030 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa351d9b640 1 -- 172.21.15.79:0/2824612530 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0019f0 con 0x7fa354070030 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa358a2a640 1 -- 172.21.15.79:0/2824612530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa354070030 msgr2=0x7fa354073bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa358a2a640 1 --2- 172.21.15.79:0/2824612530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa354070030 0x7fa354073bb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0097a0 tx=0x7fa33c02a0a0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa358a2a640 1 -- 172.21.15.79:0/2824612530 shutdown_connections 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa358a2a640 1 --2- 172.21.15.79:0/2824612530 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa354070030 0x7fa354073bb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa358a2a640 1 -- 172.21.15.79:0/2824612530 >> 172.21.15.79:0/2824612530 conn(0x7fa3540b2540 msgr2=0x7fa3540b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa358a2a640 1 -- 172.21.15.79:0/2824612530 shutdown_connections 2024-04-26T19:51:14.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.432+0000 7fa358a2a640 1 -- 172.21.15.79:0/2824612530 wait complete. 2024-04-26T19:51:14.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.434+0000 7fa358a2a640 1 Processor -- start 2024-04-26T19:51:14.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.434+0000 7fa358a2a640 1 -- start start 2024-04-26T19:51:14.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.434+0000 7fa358a2a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa354070030 0x7fa35414a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.434+0000 7fa358a2a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3540718f0 0x7fa35415b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.434+0000 7fa358a2a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa35415c1e0 0x7fa3540ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.434+0000 7fa358a2a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540ce060 con 0x7fa35415c1e0 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.434+0000 7fa358a2a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3540cdee0 con 0x7fa354070030 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.434+0000 7fa358a2a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa3540ce1e0 con 0x7fa3540718f0 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa352d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa35415c1e0 0x7fa3540ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa352d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa35415c1e0 0x7fa3540ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44262/0 (socket says 172.21.15.79:44262) 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa352d9d640 1 -- 172.21.15.79:0/1015111495 learned_addr learned my addr 172.21.15.79:0/1015111495 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa35259c640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa354070030 0x7fa35414a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa34bfff640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3540718f0 0x7fa35415b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa352d9d640 1 -- 172.21.15.79:0/1015111495 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3540718f0 msgr2=0x7fa35415b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa352d9d640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3540718f0 0x7fa35415b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa352d9d640 1 -- 172.21.15.79:0/1015111495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa354070030 msgr2=0x7fa35414a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa352d9d640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa354070030 0x7fa35414a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa352d9d640 1 -- 172.21.15.79:0/1015111495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa354153650 con 0x7fa35415c1e0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa352d9d640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa35415c1e0 0x7fa3540ae8d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00cc90 tx=0x7fa34c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa34affd640 1 -- 172.21.15.79:0/1015111495 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c017070 con 0x7fa35415c1e0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa34affd640 1 -- 172.21.15.79:0/1015111495 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa34c012400 con 0x7fa35415c1e0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35414b050 con 0x7fa35415c1e0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa34affd640 1 -- 172.21.15.79:0/1015111495 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c0127b0 con 0x7fa35415c1e0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.435+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35415e920 con 0x7fa35415c1e0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.436+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3540506c0 con 0x7fa35415c1e0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.437+0000 7fa34affd640 1 -- 172.21.15.79:0/1015111495 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa34c01a070 con 0x7fa35415c1e0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.437+0000 7fa34affd640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa33406c8f0 0x7fa33406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.437+0000 7fa34affd640 1 -- 172.21.15.79:0/1015111495 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa34c08bdb0 con 0x7fa35415c1e0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.437+0000 7fa35259c640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa33406c8f0 0x7fa33406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.438+0000 7fa35259c640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa33406c8f0 0x7fa33406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa344000f60 tx=0x7fa344009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:14.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.441+0000 7fa34affd640 1 -- 172.21.15.79:0/1015111495 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa34c059790 con 0x7fa35415c1e0 2024-04-26T19:51:14.614 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.613+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa3540cda70 con 0x7fa35415c1e0 2024-04-26T19:51:14.615 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.614+0000 7fa34affd640 1 -- 172.21.15.79:0/1015111495 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fa34c05da90 con 0x7fa35415c1e0 2024-04-26T19:51:14.615 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa33406c8f0 msgr2=0x7fa33406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa33406c8f0 0x7fa33406edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa344000f60 tx=0x7fa344009040 comp rx=0 tx=0).stop 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa35415c1e0 msgr2=0x7fa3540ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa35415c1e0 0x7fa3540ae8d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00cc90 tx=0x7fa34c00a430 comp rx=0 tx=0).stop 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 shutdown_connections 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa33406c8f0 0x7fa33406edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa35415c1e0 0x7fa3540ae8d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3540718f0 0x7fa35415b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 --2- 172.21.15.79:0/1015111495 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa354070030 0x7fa35414a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 >> 172.21.15.79:0/1015111495 conn(0x7fa3540b2540 msgr2=0x7fa3540d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 shutdown_connections 2024-04-26T19:51:14.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.616+0000 7fa358a2a640 1 -- 172.21.15.79:0/1015111495 wait complete. 2024-04-26T19:51:14.628 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:14.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.727+0000 7fda95469640 1 Processor -- start 2024-04-26T19:51:14.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.727+0000 7fda95469640 1 -- start start 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda95469640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fda900af290 0x7fda900af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda95469640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda9006fa10 con 0x7fda900aeeb0 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda95469640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda90072ea0 con 0x7fda90070160 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda95469640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fda9006f7f0 con 0x7fda900af290 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda8f7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fda900af290 0x7fda900af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda8effd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fda900aeeb0 0x7fda9006f0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35252/0 (socket says 172.21.15.79:35252) 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda8effd640 1 -- 172.21.15.79:0/3711297940 learned_addr learned my addr 172.21.15.79:0/3711297940 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda8f7fe640 1 -- 172.21.15.79:0/3711297940 >> v1:172.21.15.83:6789/0 conn(0x7fda90070160 legacy=0x7fda900b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda8f7fe640 1 -- 172.21.15.79:0/3711297940 >> v1:172.21.15.79:6789/0 conn(0x7fda900aeeb0 legacy=0x7fda9006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda8e7fc640 1 -- 172.21.15.79:0/3711297940 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 88041946 0 0) 0x7fda9006fa10 con 0x7fda900aeeb0 2024-04-26T19:51:14.729 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.728+0000 7fda8f7fe640 1 -- 172.21.15.79:0/3711297940 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda90073180 con 0x7fda900af290 2024-04-26T19:51:14.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.729+0000 7fda8f7fe640 1 --2- 172.21.15.79:0/3711297940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda900af290 0x7fda900af6c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda88009d90 tx=0x7fda8802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2794009831951 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:14.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.729+0000 7fda8e7fc640 1 -- 172.21.15.79:0/3711297940 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88007740 con 0x7fda900af290 2024-04-26T19:51:14.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.729+0000 7fda8e7fc640 1 -- 172.21.15.79:0/3711297940 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda880078e0 con 0x7fda900af290 2024-04-26T19:51:14.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.729+0000 7fda8e7fc640 1 -- 172.21.15.79:0/3711297940 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda88007cb0 con 0x7fda900af290 2024-04-26T19:51:14.731 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.729+0000 7fda95469640 1 -- 172.21.15.79:0/3711297940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda900af290 msgr2=0x7fda900af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.729+0000 7fda95469640 1 --2- 172.21.15.79:0/3711297940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda900af290 0x7fda900af6c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda88009d90 tx=0x7fda8802a660 comp rx=0 tx=0).stop 2024-04-26T19:51:14.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.729+0000 7fda95469640 1 -- 172.21.15.79:0/3711297940 shutdown_connections 2024-04-26T19:51:14.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.730+0000 7fda95469640 1 --2- 172.21.15.79:0/3711297940 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda900af290 0x7fda900af6c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.730+0000 7fda95469640 1 -- 172.21.15.79:0/3711297940 >> 172.21.15.79:0/3711297940 conn(0x7fda900687e0 msgr2=0x7fda90071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:14.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.730+0000 7fda95469640 1 -- 172.21.15.79:0/3711297940 shutdown_connections 2024-04-26T19:51:14.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.730+0000 7fda95469640 1 -- 172.21.15.79:0/3711297940 wait complete. 2024-04-26T19:51:14.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda95469640 1 Processor -- start 2024-04-26T19:51:14.732 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda95469640 1 -- start start 2024-04-26T19:51:14.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda95469640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90070160 0x7fda9013edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda95469640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda900aeeb0 0x7fda90162510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda95469640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda900af290 0x7fda90147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda95469640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fda90149380 con 0x7fda90070160 2024-04-26T19:51:14.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda95469640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda90073740 con 0x7fda900af290 2024-04-26T19:51:14.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda95469640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fda900b1b60 con 0x7fda900aeeb0 2024-04-26T19:51:14.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda8effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90070160 0x7fda9013edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda8effd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90070160 0x7fda9013edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44282/0 (socket says 172.21.15.79:44282) 2024-04-26T19:51:14.735 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.731+0000 7fda8effd640 1 -- 172.21.15.79:0/438628809 learned_addr learned my addr 172.21.15.79:0/438628809 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda87fff640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda900aeeb0 0x7fda90162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda8f7fe640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda900af290 0x7fda90147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda8effd640 1 -- 172.21.15.79:0/438628809 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda900aeeb0 msgr2=0x7fda90162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda8effd640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda900aeeb0 0x7fda90162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda8effd640 1 -- 172.21.15.79:0/438628809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda900af290 msgr2=0x7fda90147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda8effd640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda900af290 0x7fda90147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda8effd640 1 -- 172.21.15.79:0/438628809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda90160260 con 0x7fda90070160 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda8effd640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90070160 0x7fda9013edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c4a0 tx=0x7fda8000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda877fe640 1 -- 172.21.15.79:0/438628809 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80018070 con 0x7fda90070160 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda877fe640 1 -- 172.21.15.79:0/438628809 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda800074e0 con 0x7fda90070160 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda877fe640 1 -- 172.21.15.79:0/438628809 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda80004a20 con 0x7fda90070160 2024-04-26T19:51:14.736 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda90160540 con 0x7fda90070160 2024-04-26T19:51:14.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.732+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda90160b00 con 0x7fda90070160 2024-04-26T19:51:14.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.733+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda9006e080 con 0x7fda90070160 2024-04-26T19:51:14.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.733+0000 7fda877fe640 1 -- 172.21.15.79:0/438628809 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda8001b070 con 0x7fda90070160 2024-04-26T19:51:14.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.734+0000 7fda877fe640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda7006c920 0x7fda7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:14.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.734+0000 7fda87fff640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda7006c920 0x7fda7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:14.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.734+0000 7fda877fe640 1 -- 172.21.15.79:0/438628809 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fda8008b280 con 0x7fda90070160 2024-04-26T19:51:14.737 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.734+0000 7fda87fff640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda7006c920 0x7fda7006ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda90069670 tx=0x7fda78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:14.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.738+0000 7fda877fe640 1 -- 172.21.15.79:0/438628809 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda9006e080 con 0x7fda90070160 2024-04-26T19:51:14.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.912+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fda900506c0 con 0x7fda90070160 2024-04-26T19:51:14.913 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.912+0000 7fda877fe640 1 -- 172.21.15.79:0/438628809 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fda800592b0 con 0x7fda90070160 2024-04-26T19:51:14.913 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda7006c920 msgr2=0x7fda7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda7006c920 0x7fda7006ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda90069670 tx=0x7fda78009040 comp rx=0 tx=0).stop 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90070160 msgr2=0x7fda9013edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90070160 0x7fda9013edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fda8000c4a0 tx=0x7fda8000c960 comp rx=0 tx=0).stop 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 shutdown_connections 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda7006c920 0x7fda7006ede0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda900af290 0x7fda90147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda900aeeb0 0x7fda90162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 --2- 172.21.15.79:0/438628809 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda90070160 0x7fda9013edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 >> 172.21.15.79:0/438628809 conn(0x7fda900687e0 msgr2=0x7fda900b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 shutdown_connections 2024-04-26T19:51:14.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:14.915+0000 7fda95469640 1 -- 172.21.15.79:0/438628809 wait complete. 2024-04-26T19:51:14.926 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:15.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.022+0000 7f9b84064640 1 Processor -- start 2024-04-26T19:51:15.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.022+0000 7f9b84064640 1 -- start start 2024-04-26T19:51:15.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b84064640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 0x7f9b7c161360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b84064640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b7c0d15d0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b84064640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b7c0d2f80 con 0x7f9b7c0b3d80 2024-04-26T19:51:15.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b84064640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c161a20 con 0x7f9b7c160ed0 2024-04-26T19:51:15.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b815ff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9b7c0b4160 0x7f9b7c0aa4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35268/0 (socket says 172.21.15.79:35268) 2024-04-26T19:51:15.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b815ff640 1 -- 172.21.15.79:0/2881812568 learned_addr learned my addr 172.21.15.79:0/2881812568 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:15.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b82601640 1 --2- 172.21.15.79:0/2881812568 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 0x7f9b7c161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b80dfe640 1 -- 172.21.15.79:0/2881812568 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 271212297 0 0) 0x7f9b7c0d15d0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b80dfe640 1 -- 172.21.15.79:0/2881812568 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b68003520 con 0x7f9b7c0b4160 2024-04-26T19:51:15.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b80dfe640 1 -- 172.21.15.79:0/2881812568 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2214236244 0 0) 0x7f9b7c0d2f80 con 0x7f9b7c0b3d80 2024-04-26T19:51:15.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.023+0000 7f9b80dfe640 1 -- 172.21.15.79:0/2881812568 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b7c0d15d0 con 0x7f9b7c0b3d80 2024-04-26T19:51:15.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.024+0000 7f9b82601640 1 -- 172.21.15.79:0/2881812568 >> v1:172.21.15.83:6789/0 conn(0x7f9b7c0b3d80 legacy=0x7f9b7c0d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.024+0000 7f9b82601640 1 -- 172.21.15.79:0/2881812568 >> v1:172.21.15.79:6789/0 conn(0x7f9b7c0b4160 legacy=0x7f9b7c0aa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.024+0000 7f9b82601640 1 -- 172.21.15.79:0/2881812568 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c161d00 con 0x7f9b7c160ed0 2024-04-26T19:51:15.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.024+0000 7f9b82601640 1 --2- 172.21.15.79:0/2881812568 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 0x7f9b7c161360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b740097d0 tx=0x7f9b7402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2499bfc341bcee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.024+0000 7f9b80dfe640 1 -- 172.21.15.79:0/2881812568 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b74037070 con 0x7f9b7c160ed0 2024-04-26T19:51:15.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.024+0000 7f9b80dfe640 1 -- 172.21.15.79:0/2881812568 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b74032440 con 0x7f9b7c160ed0 2024-04-26T19:51:15.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.024+0000 7f9b80dfe640 1 -- 172.21.15.79:0/2881812568 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b740327f0 con 0x7f9b7c160ed0 2024-04-26T19:51:15.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.025+0000 7f9b84064640 1 -- 172.21.15.79:0/2881812568 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 msgr2=0x7f9b7c161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.025+0000 7f9b84064640 1 --2- 172.21.15.79:0/2881812568 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 0x7f9b7c161360 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9b740097d0 tx=0x7f9b7402a210 comp rx=0 tx=0).stop 2024-04-26T19:51:15.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.025+0000 7f9b84064640 1 -- 172.21.15.79:0/2881812568 shutdown_connections 2024-04-26T19:51:15.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.025+0000 7f9b84064640 1 --2- 172.21.15.79:0/2881812568 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 0x7f9b7c161360 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.025+0000 7f9b84064640 1 -- 172.21.15.79:0/2881812568 >> 172.21.15.79:0/2881812568 conn(0x7f9b7c0b2720 msgr2=0x7f9b7c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:15.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.025+0000 7f9b84064640 1 -- 172.21.15.79:0/2881812568 shutdown_connections 2024-04-26T19:51:15.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.025+0000 7f9b84064640 1 -- 172.21.15.79:0/2881812568 wait complete. 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.026+0000 7f9b84064640 1 Processor -- start 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.026+0000 7f9b84064640 1 -- start start 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b84064640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b7c0b3d80 0x7f9b7c1420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b84064640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b7c0b4160 0x7f9b7c157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b84064640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 0x7f9b7c13c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b84064640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c0d2780 con 0x7f9b7c0b4160 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b84064640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9b7c0d1b50 con 0x7f9b7c0b3d80 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b84064640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9b7c0d2900 con 0x7f9b7c160ed0 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b815ff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b7c0b4160 0x7f9b7c157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b815ff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b7c0b4160 0x7f9b7c157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44298/0 (socket says 172.21.15.79:44298) 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b815ff640 1 -- 172.21.15.79:0/1846086363 learned_addr learned my addr 172.21.15.79:0/1846086363 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b82601640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 0x7f9b7c13c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b81e00640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b7c0b3d80 0x7f9b7c1420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b815ff640 1 -- 172.21.15.79:0/1846086363 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 msgr2=0x7f9b7c13c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b815ff640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 0x7f9b7c13c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b815ff640 1 -- 172.21.15.79:0/1846086363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b7c0b3d80 msgr2=0x7f9b7c1420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b815ff640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b7c0b3d80 0x7f9b7c1420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.027+0000 7f9b815ff640 1 -- 172.21.15.79:0/1846086363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b7c15cdc0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.028+0000 7f9b815ff640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b7c0b4160 0x7f9b7c157f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000c410 tx=0x7f9b7000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.028+0000 7f9b6e7fc640 1 -- 172.21.15.79:0/1846086363 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b70017070 con 0x7f9b7c0b4160 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.028+0000 7f9b6e7fc640 1 -- 172.21.15.79:0/1846086363 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b70005ce0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.028+0000 7f9b6e7fc640 1 -- 172.21.15.79:0/1846086363 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b700072d0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.028+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b7c15cae0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.028+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b7c1418f0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.029+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b7c0506c0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.029+0000 7f9b6e7fc640 1 -- 172.21.15.79:0/1846086363 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b7001a070 con 0x7f9b7c0b4160 2024-04-26T19:51:15.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.029+0000 7f9b6e7fc640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9b6006c920 0x7f9b6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.030+0000 7f9b6e7fc640 1 -- 172.21.15.79:0/1846086363 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9b7008b200 con 0x7f9b7c0b4160 2024-04-26T19:51:15.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.030+0000 7f9b81e00640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9b6006c920 0x7f9b6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.030+0000 7f9b81e00640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9b6006c920 0x7f9b6006ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001e00 tx=0x7f9b78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.034+0000 7f9b6e7fc640 1 -- 172.21.15.79:0/1846086363 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9b70058bd0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.215+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b7c06f4d0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.216+0000 7f9b6e7fc640 1 -- 172.21.15.79:0/1846086363 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7f9b7005ced0 con 0x7f9b7c0b4160 2024-04-26T19:51:15.217 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:15.217 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:15.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.218+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9b6006c920 msgr2=0x7f9b6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.218+0000 7f9b84064640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9b6006c920 0x7f9b6006ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b78001e00 tx=0x7f9b78009040 comp rx=0 tx=0).stop 2024-04-26T19:51:15.219 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.218+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b7c0b4160 msgr2=0x7f9b7c157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.218+0000 7f9b84064640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b7c0b4160 0x7f9b7c157f80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9b7000c410 tx=0x7f9b7000c8d0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.219+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 shutdown_connections 2024-04-26T19:51:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.219+0000 7f9b84064640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9b6006c920 0x7f9b6006ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.219+0000 7f9b84064640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9b7c160ed0 0x7f9b7c13c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.219+0000 7f9b84064640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9b7c0b4160 0x7f9b7c157f80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.219+0000 7f9b84064640 1 --2- 172.21.15.79:0/1846086363 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9b7c0b3d80 0x7f9b7c1420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.219+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 >> 172.21.15.79:0/1846086363 conn(0x7f9b7c0b2720 msgr2=0x7f9b7c0aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.219+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 shutdown_connections 2024-04-26T19:51:15.220 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.219+0000 7f9b84064640 1 -- 172.21.15.79:0/1846086363 wait complete. 2024-04-26T19:51:15.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:15.232 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:15.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.331+0000 7fa2c4922640 1 Processor -- start 2024-04-26T19:51:15.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.331+0000 7fa2c4922640 1 -- start start 2024-04-26T19:51:15.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.332+0000 7fa2c4922640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa2c0070030 0x7fa2c0073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.332+0000 7fa2c4922640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c0072620 con 0x7fa2c00718f0 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.332+0000 7fa2c4922640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2c00730d0 con 0x7fa2c015c1e0 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.332+0000 7fa2c4922640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fa2c00733b0 con 0x7fa2c0070030 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.333+0000 7fa2bdd9b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa2c0070030 0x7fa2c0073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.333+0000 7fa2bed9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa2c00718f0 0x7fa2c0071d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35282/0 (socket says 172.21.15.79:35282) 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.333+0000 7fa2bdd9b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa2c0070030 0x7fa2c0073bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44300/0 (socket says 172.21.15.79:44300) 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.333+0000 7fa2bed9d640 1 -- 172.21.15.79:0/551955418 learned_addr learned my addr 172.21.15.79:0/551955418 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.333+0000 7fa2bdd9b640 1 -- 172.21.15.79:0/551955418 >> v1:172.21.15.83:6789/0 conn(0x7fa2c015c1e0 legacy=0x7fa2c006f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.333+0000 7fa2bdd9b640 1 -- 172.21.15.79:0/551955418 >> v1:172.21.15.79:6789/0 conn(0x7fa2c00718f0 legacy=0x7fa2c0071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.333+0000 7fa2bdd9b640 1 -- 172.21.15.79:0/551955418 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c0073690 con 0x7fa2c0070030 2024-04-26T19:51:15.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.333+0000 7fa2bd59a640 1 -- 172.21.15.79:0/551955418 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 627299759 0 0) 0x7fa2c0072620 con 0x7fa2c00718f0 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.333+0000 7fa2bdd9b640 1 --2- 172.21.15.79:0/551955418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c0070030 0x7fa2c0073bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009d90 tx=0x7fa2b402a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1886f9a628fb994 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2bd59a640 1 -- 172.21.15.79:0/551955418 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4007740 con 0x7fa2c0070030 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2bd59a640 1 -- 172.21.15.79:0/551955418 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b40078e0 con 0x7fa2c0070030 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2bd59a640 1 -- 172.21.15.79:0/551955418 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4007cb0 con 0x7fa2c0070030 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2c4922640 1 -- 172.21.15.79:0/551955418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c0070030 msgr2=0x7fa2c0073bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2c4922640 1 --2- 172.21.15.79:0/551955418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c0070030 0x7fa2c0073bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009d90 tx=0x7fa2b402a580 comp rx=0 tx=0).stop 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2c4922640 1 -- 172.21.15.79:0/551955418 shutdown_connections 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2c4922640 1 --2- 172.21.15.79:0/551955418 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c0070030 0x7fa2c0073bb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2c4922640 1 -- 172.21.15.79:0/551955418 >> 172.21.15.79:0/551955418 conn(0x7fa2c00b2540 msgr2=0x7fa2c00b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2c4922640 1 -- 172.21.15.79:0/551955418 shutdown_connections 2024-04-26T19:51:15.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.334+0000 7fa2c4922640 1 -- 172.21.15.79:0/551955418 wait complete. 2024-04-26T19:51:15.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2c4922640 1 Processor -- start 2024-04-26T19:51:15.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2c4922640 1 -- start start 2024-04-26T19:51:15.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2c4922640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa2c0070030 0x7fa2c015c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2c4922640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c00718f0 0x7fa2c01f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2c4922640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c015c1e0 0x7fa2c01f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2c4922640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00ce060 con 0x7fa2c00718f0 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2c4922640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c00cdee0 con 0x7fa2c015c1e0 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2c4922640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c00ce1e0 con 0x7fa2c0070030 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2bdd9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c00718f0 0x7fa2c01f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2bdd9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c00718f0 0x7fa2c01f4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44308/0 (socket says 172.21.15.79:44308) 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2bdd9b640 1 -- 172.21.15.79:0/636868532 learned_addr learned my addr 172.21.15.79:0/636868532 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2bed9d640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c015c1e0 0x7fa2c01f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.336+0000 7fa2be59c640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa2c0070030 0x7fa2c015c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2bdd9b640 1 -- 172.21.15.79:0/636868532 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa2c0070030 msgr2=0x7fa2c015c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2bdd9b640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa2c0070030 0x7fa2c015c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2bdd9b640 1 -- 172.21.15.79:0/636868532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c015c1e0 msgr2=0x7fa2c01f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2bdd9b640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c015c1e0 0x7fa2c01f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2bdd9b640 1 -- 172.21.15.79:0/636868532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c015b0f0 con 0x7fa2c00718f0 2024-04-26T19:51:15.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2bdd9b640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c00718f0 0x7fa2c01f4790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009d90 tx=0x7fa2b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2aeffd640 1 -- 172.21.15.79:0/636868532 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4040070 con 0x7fa2c00718f0 2024-04-26T19:51:15.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2aeffd640 1 -- 172.21.15.79:0/636868532 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2b4031c40 con 0x7fa2c00718f0 2024-04-26T19:51:15.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c015a570 con 0x7fa2c00718f0 2024-04-26T19:51:15.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2aeffd640 1 -- 172.21.15.79:0/636868532 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b403c420 con 0x7fa2c00718f0 2024-04-26T19:51:15.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.337+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c015ab30 con 0x7fa2c00718f0 2024-04-26T19:51:15.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.338+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c00506c0 con 0x7fa2c00718f0 2024-04-26T19:51:15.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.338+0000 7fa2aeffd640 1 -- 172.21.15.79:0/636868532 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2b4039040 con 0x7fa2c00718f0 2024-04-26T19:51:15.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.339+0000 7fa2aeffd640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2a006c8f0 0x7fa2a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.339+0000 7fa2aeffd640 1 -- 172.21.15.79:0/636868532 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa2b40ad410 con 0x7fa2c00718f0 2024-04-26T19:51:15.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.339+0000 7fa2be59c640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2a006c8f0 0x7fa2a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.339+0000 7fa2be59c640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2a006c8f0 0x7fa2a006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00741a0 tx=0x7fa2b8009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.343+0000 7fa2aeffd640 1 -- 172.21.15.79:0/636868532 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa2b407ae80 con 0x7fa2c00718f0 2024-04-26T19:51:15.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.518+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2c0072a60 con 0x7fa2c00718f0 2024-04-26T19:51:15.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.518+0000 7fa2aeffd640 1 -- 172.21.15.79:0/636868532 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fa2b407f180 con 0x7fa2c00718f0 2024-04-26T19:51:15.519 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2a006c8f0 msgr2=0x7fa2a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2a006c8f0 0x7fa2a006edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa2c00741a0 tx=0x7fa2b8009060 comp rx=0 tx=0).stop 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c00718f0 msgr2=0x7fa2c01f4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c00718f0 0x7fa2c01f4790 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009d90 tx=0x7fa2b4004a40 comp rx=0 tx=0).stop 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 shutdown_connections 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2a006c8f0 0x7fa2a006edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2c015c1e0 0x7fa2c01f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa2c00718f0 0x7fa2c01f4790 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 --2- 172.21.15.79:0/636868532 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa2c0070030 0x7fa2c015c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 >> 172.21.15.79:0/636868532 conn(0x7fa2c00b2540 msgr2=0x7fa2c00b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:15.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 shutdown_connections 2024-04-26T19:51:15.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.520+0000 7fa2c4922640 1 -- 172.21.15.79:0/636868532 wait complete. 2024-04-26T19:51:15.532 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:15.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.631+0000 7f4e22f4d640 1 Processor -- start 2024-04-26T19:51:15.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.631+0000 7f4e22f4d640 1 -- start start 2024-04-26T19:51:15.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.632+0000 7f4e22f4d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4e1c146bc0 0x7f4e1c0b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.632+0000 7f4e22f4d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4e1c0d2d40 0x7f4e1c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.632+0000 7f4e22f4d640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e1c0b7c60 con 0x7f4e1c0b91d0 2024-04-26T19:51:15.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.632+0000 7f4e22f4d640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f4e1c0b7ed0 con 0x7f4e1c146bc0 2024-04-26T19:51:15.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.632+0000 7f4e22f4d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4e1c0b81b0 con 0x7f4e1c0d2d40 2024-04-26T19:51:15.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.632+0000 7f4e20ce9640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4e1c146bc0 0x7f4e1c0b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.632+0000 7f4e214ea640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4e1c0b91d0 0x7f4e1c0b3480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35292/0 (socket says 172.21.15.79:35292) 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.632+0000 7f4e214ea640 1 -- 172.21.15.79:0/134398301 learned_addr learned my addr 172.21.15.79:0/134398301 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.633+0000 7f4e1bfff640 1 --2- 172.21.15.79:0/134398301 >> v2:172.21.15.83:3300/0 conn(0x7f4e1c0d2d40 0x7f4e1c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.633+0000 7f4e20ce9640 1 -- 172.21.15.79:0/134398301 >> v2:172.21.15.83:3300/0 conn(0x7f4e1c0d2d40 msgr2=0x7f4e1c0b8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.633+0000 7f4e20ce9640 1 --2- 172.21.15.79:0/134398301 >> v2:172.21.15.83:3300/0 conn(0x7f4e1c0d2d40 0x7f4e1c0b8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.633+0000 7f4e20ce9640 1 -- 172.21.15.79:0/134398301 >> v1:172.21.15.79:6789/0 conn(0x7f4e1c0b91d0 legacy=0x7f4e1c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.633+0000 7f4e20ce9640 1 -- 172.21.15.79:0/134398301 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c0b8490 con 0x7f4e1c146bc0 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.633+0000 7f4e1b7fe640 1 -- 172.21.15.79:0/134398301 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4068573412 0 0) 0x7f4e1c0b7c60 con 0x7f4e1c0b91d0 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.633+0000 7f4e20ce9640 1 --2- 172.21.15.79:0/134398301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 0x7f4e1c0b6e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ab650f40429160 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e1b7fe640 1 -- 172.21.15.79:0/134398301 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c037070 con 0x7f4e1c146bc0 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e1b7fe640 1 -- 172.21.15.79:0/134398301 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e0c02ae50 con 0x7f4e1c146bc0 2024-04-26T19:51:15.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e1b7fe640 1 -- 172.21.15.79:0/134398301 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c032660 con 0x7f4e1c146bc0 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e22f4d640 1 -- 172.21.15.79:0/134398301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 msgr2=0x7f4e1c0b6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e22f4d640 1 --2- 172.21.15.79:0/134398301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 0x7f4e1c0b6e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c02a140 comp rx=0 tx=0).stop 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e22f4d640 1 -- 172.21.15.79:0/134398301 shutdown_connections 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e22f4d640 1 --2- 172.21.15.79:0/134398301 >> v2:172.21.15.83:3300/0 conn(0x7f4e1c0d2d40 0x7f4e1c0b8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e22f4d640 1 --2- 172.21.15.79:0/134398301 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 0x7f4e1c0b6e80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e22f4d640 1 -- 172.21.15.79:0/134398301 >> 172.21.15.79:0/134398301 conn(0x7f4e1c0b4500 msgr2=0x7f4e1c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e22f4d640 1 -- 172.21.15.79:0/134398301 shutdown_connections 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.634+0000 7f4e22f4d640 1 -- 172.21.15.79:0/134398301 wait complete. 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.635+0000 7f4e22f4d640 1 Processor -- start 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.635+0000 7f4e22f4d640 1 -- start start 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.636+0000 7f4e22f4d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e1c0d2d40 0x7f4e1c13f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.636+0000 7f4e22f4d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 0x7f4e1c1e3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.636+0000 7f4e22f4d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4e1c1e4040 0x7f4e1c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.636+0000 7f4e22f4d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c0b73c0 con 0x7f4e1c146bc0 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.636+0000 7f4e22f4d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4e1c146fa0 con 0x7f4e1c0d2d40 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.636+0000 7f4e22f4d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4e1c0b7540 con 0x7f4e1c1e4040 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.636+0000 7f4e1bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 0x7f4e1c1e3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.636+0000 7f4e1bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 0x7f4e1c1e3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44332/0 (socket says 172.21.15.79:44332) 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.636+0000 7f4e1bfff640 1 -- 172.21.15.79:0/3665112210 learned_addr learned my addr 172.21.15.79:0/3665112210 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e20ce9640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e1c0d2d40 0x7f4e1c13f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e214ea640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4e1c1e4040 0x7f4e1c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e1bfff640 1 -- 172.21.15.79:0/3665112210 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4e1c1e4040 msgr2=0x7f4e1c0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e1bfff640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4e1c1e4040 0x7f4e1c0b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e1bfff640 1 -- 172.21.15.79:0/3665112210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e1c0d2d40 msgr2=0x7f4e1c13f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e1bfff640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e1c0d2d40 0x7f4e1c13f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e1bfff640 1 -- 172.21.15.79:0/3665112210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e1c150c90 con 0x7f4e1c146bc0 2024-04-26T19:51:15.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e20ce9640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e1c0d2d40 0x7f4e1c13f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:15.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e1bfff640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 0x7f4e1c1e3b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000d7d0 tx=0x7f4e1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e197fa640 1 -- 172.21.15.79:0/3665112210 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000b6c0 con 0x7f4e1c146bc0 2024-04-26T19:51:15.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e197fa640 1 -- 172.21.15.79:0/3665112210 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e10019070 con 0x7f4e1c146bc0 2024-04-26T19:51:15.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e1c150f70 con 0x7f4e1c146bc0 2024-04-26T19:51:15.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e197fa640 1 -- 172.21.15.79:0/3665112210 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1000bac0 con 0x7f4e1c146bc0 2024-04-26T19:51:15.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.637+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e1c151530 con 0x7f4e1c146bc0 2024-04-26T19:51:15.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.638+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e1c0506c0 con 0x7f4e1c146bc0 2024-04-26T19:51:15.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.638+0000 7f4e197fa640 1 -- 172.21.15.79:0/3665112210 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4e1000bc60 con 0x7f4e1c146bc0 2024-04-26T19:51:15.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.639+0000 7f4e197fa640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4e0806c8f0 0x7f4e0806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.639+0000 7f4e197fa640 1 -- 172.21.15.79:0/3665112210 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4e1008c3d0 con 0x7f4e1c146bc0 2024-04-26T19:51:15.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.639+0000 7f4e20ce9640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4e0806c8f0 0x7f4e0806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.639+0000 7f4e20ce9640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4e0806c8f0 0x7f4e0806edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c034000 tx=0x7f4e0c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.643+0000 7f4e197fa640 1 -- 172.21.15.79:0/3665112210 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4e1005a490 con 0x7f4e1c146bc0 2024-04-26T19:51:15.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.817+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4e1c06f1c0 con 0x7f4e1c146bc0 2024-04-26T19:51:15.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.818+0000 7f4e197fa640 1 -- 172.21.15.79:0/3665112210 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f4e10059d90 con 0x7f4e1c146bc0 2024-04-26T19:51:15.818 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4e0806c8f0 msgr2=0x7f4e0806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4e0806c8f0 0x7f4e0806edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c034000 tx=0x7f4e0c02e000 comp rx=0 tx=0).stop 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 msgr2=0x7f4e1c1e3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 0x7f4e1c1e3b00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4e1000d7d0 tx=0x7f4e1000dc90 comp rx=0 tx=0).stop 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 shutdown_connections 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4e0806c8f0 0x7f4e0806edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4e1c1e4040 0x7f4e1c0b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4e1c146bc0 0x7f4e1c1e3b00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 --2- 172.21.15.79:0/3665112210 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4e1c0d2d40 0x7f4e1c13f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.821 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 >> 172.21.15.79:0/3665112210 conn(0x7f4e1c0b4500 msgr2=0x7f4e1c0aa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:15.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 shutdown_connections 2024-04-26T19:51:15.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.820+0000 7f4e22f4d640 1 -- 172.21.15.79:0/3665112210 wait complete. 2024-04-26T19:51:15.831 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:15.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.930+0000 7f22e25f0640 1 Processor -- start 2024-04-26T19:51:15.931 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.930+0000 7f22e25f0640 1 -- start start 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22e25f0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f22dc06ca60 0x7f22dc0d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22e25f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc148980 0x7f22dc0b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22e25f0640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22dc0d2580 con 0x7f22dc1482e0 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22e25f0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f22dc0d2860 con 0x7f22dc06ca60 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22e25f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0b6a80 con 0x7f22dc148980 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22dbfff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f22dc1482e0 0x7f22dc06c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35296/0 (socket says 172.21.15.79:35296) 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22dbfff640 1 -- 172.21.15.79:0/4198639949 learned_addr learned my addr 172.21.15.79:0/4198639949 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22db7fe640 1 --2- 172.21.15.79:0/4198639949 >> v2:172.21.15.83:3300/0 conn(0x7f22dc06ca60 0x7f22dc0d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 533394489 0 0) 0x7f22dc0d2580 con 0x7f22dc1482e0 2024-04-26T19:51:15.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22c4003520 con 0x7f22dc1482e0 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.931+0000 7f22e0b8d640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc148980 0x7f22dc0b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3062579553 0 0) 0x7f22c4003520 con 0x7f22dc1482e0 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc148980 msgr2=0x7f22dc0b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc148980 0x7f22dc0b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 >> v2:172.21.15.83:3300/0 conn(0x7f22dc06ca60 msgr2=0x7f22dc0d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 --2- 172.21.15.79:0/4198639949 >> v2:172.21.15.83:3300/0 conn(0x7f22dc06ca60 0x7f22dc0d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc0b6d60 con 0x7f22dc1482e0 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f22cc001670 con 0x7f22dc1482e0 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 >> v1:172.21.15.79:6789/0 conn(0x7f22dc1482e0 legacy=0x7f22dc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c40039b0 0x7f22c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc148980 0x7f22c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0d2860 con 0x7f22c4004800 2024-04-26T19:51:15.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc0b6a80 con 0x7f22c40039b0 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f22c400b9c0 con 0x7f22dc148980 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22db7fe640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22e0b8d640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc148980 0x7f22c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22dbfff640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c40039b0 0x7f22c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22db7fe640 1 -- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc148980 msgr2=0x7f22c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22db7fe640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc148980 0x7f22c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22db7fe640 1 -- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c40039b0 msgr2=0x7f22c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22db7fe640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22c40039b0 0x7f22c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22db7fe640 1 -- 172.21.15.79:0/4198639949 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c4003520 con 0x7f22c4004800 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.932+0000 7f22e0b8d640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc148980 0x7f22c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.933+0000 7f22db7fe640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22c4004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009ac0 tx=0x7f22c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbad21d0185ce998 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.933+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0036070 con 0x7f22c4004800 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.933+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22d0031400 con 0x7f22c4004800 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.933+0000 7f22daffd640 1 -- 172.21.15.79:0/4198639949 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0009690 con 0x7f22c4004800 2024-04-26T19:51:15.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.933+0000 7f22e25f0640 1 -- 172.21.15.79:0/4198639949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 msgr2=0x7f22c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.933+0000 7f22e25f0640 1 --2- 172.21.15.79:0/4198639949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22c4004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009ac0 tx=0x7f22c4005830 comp rx=0 tx=0).stop 2024-04-26T19:51:15.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.933+0000 7f22e0b8d640 1 -- 172.21.15.79:0/4198639949 reap_dead start 2024-04-26T19:51:15.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.933+0000 7f22e25f0640 1 -- 172.21.15.79:0/4198639949 shutdown_connections 2024-04-26T19:51:15.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.933+0000 7f22e25f0640 1 -- 172.21.15.79:0/4198639949 >> 172.21.15.79:0/4198639949 conn(0x7f22dc073b40 msgr2=0x7f22dc073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:15.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.934+0000 7f22e25f0640 1 -- 172.21.15.79:0/4198639949 shutdown_connections 2024-04-26T19:51:15.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.934+0000 7f22e25f0640 1 -- 172.21.15.79:0/4198639949 wait complete. 2024-04-26T19:51:15.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.935+0000 7f22e25f0640 1 Processor -- start 2024-04-26T19:51:15.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.935+0000 7f22e25f0640 1 -- start start 2024-04-26T19:51:15.940 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.935+0000 7f22e25f0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22dc1f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22e25f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22dc13f420 0x7f22dc13bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22e25f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc160e50 0x7f22dc1463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22e25f0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc149100 con 0x7f22c4004800 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22e25f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1486c0 con 0x7f22dc13f420 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22e25f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0b7280 con 0x7f22dc160e50 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22dc1f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22dbfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22dc1f4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44352/0 (socket says 172.21.15.79:44352) 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22dbfff640 1 -- 172.21.15.79:0/811360157 learned_addr learned my addr 172.21.15.79:0/811360157 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22dbfff640 1 -- 172.21.15.79:0/811360157 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc160e50 msgr2=0x7f22dc1463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22dbfff640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc160e50 0x7f22dc1463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22dbfff640 1 -- 172.21.15.79:0/811360157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22dc13f420 msgr2=0x7f22dc13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22db7fe640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22dc13f420 0x7f22dc13bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22dbfff640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22dc13f420 0x7f22dc13bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:15.941 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22dbfff640 1 -- 172.21.15.79:0/811360157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc1f5de0 con 0x7f22c4004800 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22dbfff640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22dc1f4e60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f22cc001200 tx=0x7f22cc00ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22d8ff9640 1 -- 172.21.15.79:0/811360157 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00ee00 con 0x7f22c4004800 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22d8ff9640 1 -- 172.21.15.79:0/811360157 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22cc018070 con 0x7f22c4004800 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc1f5540 con 0x7f22c4004800 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22d8ff9640 1 -- 172.21.15.79:0/811360157 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00caf0 con 0x7f22c4004800 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.936+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1f5b00 con 0x7f22c4004800 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.937+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc0506c0 con 0x7f22c4004800 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.938+0000 7f22d8ff9640 1 -- 172.21.15.79:0/811360157 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f22cc00cc90 con 0x7f22c4004800 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.938+0000 7f22d8ff9640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f22c806c920 0x7f22c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.938+0000 7f22d8ff9640 1 -- 172.21.15.79:0/811360157 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f22cc08bd60 con 0x7f22c4004800 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.938+0000 7f22db7fe640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f22c806c920 0x7f22c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:15.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.938+0000 7f22db7fe640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f22c806c920 0x7f22c806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22d0033000 tx=0x7f22d0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:15.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:15.942+0000 7f22d8ff9640 1 -- 172.21.15.79:0/811360157 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f22cc059e20 con 0x7f22c4004800 2024-04-26T19:51:16.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.118+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22dc0b9120 con 0x7f22c4004800 2024-04-26T19:51:16.119 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.119+0000 7f22d8ff9640 1 -- 172.21.15.79:0/811360157 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f22cc059720 con 0x7f22c4004800 2024-04-26T19:51:16.119 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:16.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f22c806c920 msgr2=0x7f22c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f22c806c920 0x7f22c806ede0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22d0033000 tx=0x7f22d0008030 comp rx=0 tx=0).stop 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 msgr2=0x7f22dc1f4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22dc1f4e60 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f22cc001200 tx=0x7f22cc00ebb0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 shutdown_connections 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f22c806c920 0x7f22c806ede0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22dc160e50 0x7f22dc1463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f22dc13f420 0x7f22dc13bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 --2- 172.21.15.79:0/811360157 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22c4004800 0x7f22dc1f4e60 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 >> 172.21.15.79:0/811360157 conn(0x7f22dc073b40 msgr2=0x7f22dc06fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 shutdown_connections 2024-04-26T19:51:16.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.121+0000 7f22e25f0640 1 -- 172.21.15.79:0/811360157 wait complete. 2024-04-26T19:51:16.132 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:16.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.231+0000 7f3df44d1640 1 Processor -- start 2024-04-26T19:51:16.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.231+0000 7f3df44d1640 1 -- start start 2024-04-26T19:51:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df44d1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec0b8f60 0x7f3dec06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df44d1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3dec0d1830 0x7f3dec06c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df44d1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dec0d1dd0 con 0x7f3dec15fa20 2024-04-26T19:51:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df44d1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3dec0d2950 con 0x7f3dec0d1830 2024-04-26T19:51:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df44d1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3dec0d20b0 con 0x7f3dec0b8f60 2024-04-26T19:51:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df2a6e640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3dec15fa20 0x7f3dec162ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35308/0 (socket says 172.21.15.79:35308) 2024-04-26T19:51:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df2a6e640 1 -- 172.21.15.79:0/883486675 learned_addr learned my addr 172.21.15.79:0/883486675 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:16.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260571077 0 0) 0x7f3dec0d1dd0 con 0x7f3dec15fa20 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dd8003520 con 0x7f3dec15fa20 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df1a6c640 1 --2- 172.21.15.79:0/883486675 >> v2:172.21.15.83:3300/0 conn(0x7f3dec0d1830 0x7f3dec06c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df226d640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec0b8f60 0x7f3dec06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 292917901 0 0) 0x7f3dd8003520 con 0x7f3dec15fa20 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec0b8f60 msgr2=0x7f3dec06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec0b8f60 0x7f3dec06c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 >> v2:172.21.15.83:3300/0 conn(0x7f3dec0d1830 msgr2=0x7f3dec06c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 --2- 172.21.15.79:0/883486675 >> v2:172.21.15.83:3300/0 conn(0x7f3dec0d1830 0x7f3dec06c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec0d2390 con 0x7f3dec15fa20 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f3de40019f0 con 0x7f3dec15fa20 2024-04-26T19:51:16.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 >> v1:172.21.15.79:6789/0 conn(0x7f3dec15fa20 legacy=0x7f3dec162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dd80039b0 0x7f3dd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec0b8f60 0x7f3dd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8004d40 0x7f3dd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec0d2950 con 0x7f3dd80039b0 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec0d20b0 con 0x7f3dd8004d40 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd800baf0 con 0x7f3dec0b8f60 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.232+0000 7f3df226d640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dd80039b0 0x7f3dd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.233+0000 7f3df1a6c640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec0b8f60 0x7f3dd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.233+0000 7f3df2a6e640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8004d40 0x7f3dd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.233+0000 7f3df226d640 1 -- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec0b8f60 msgr2=0x7f3dd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:16.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.233+0000 7f3df226d640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec0b8f60 0x7f3dd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.233+0000 7f3df226d640 1 -- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8004d40 msgr2=0x7f3dd80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.233+0000 7f3df226d640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8004d40 0x7f3dd80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.233+0000 7f3df226d640 1 -- 172.21.15.79:0/883486675 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd8003520 con 0x7f3dd80039b0 2024-04-26T19:51:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.233+0000 7f3df2a6e640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd8004d40 0x7f3dd80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.233+0000 7f3df226d640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dd80039b0 0x7f3dd8003dd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ba30 tx=0x7f3dd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322f9b68de4a5e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8030070 con 0x7f3dd80039b0 2024-04-26T19:51:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3de802ade0 con 0x7f3dd80039b0 2024-04-26T19:51:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df126b640 1 -- 172.21.15.79:0/883486675 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de800b600 con 0x7f3dd80039b0 2024-04-26T19:51:16.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df44d1640 1 -- 172.21.15.79:0/883486675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dd80039b0 msgr2=0x7f3dd8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df44d1640 1 --2- 172.21.15.79:0/883486675 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dd80039b0 0x7f3dd8003dd0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ba30 tx=0x7f3dd8005830 comp rx=0 tx=0).stop 2024-04-26T19:51:16.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df2a6e640 1 -- 172.21.15.79:0/883486675 reap_dead start 2024-04-26T19:51:16.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df44d1640 1 -- 172.21.15.79:0/883486675 shutdown_connections 2024-04-26T19:51:16.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df44d1640 1 -- 172.21.15.79:0/883486675 >> 172.21.15.79:0/883486675 conn(0x7f3dec073b40 msgr2=0x7f3dec073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:16.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df44d1640 1 -- 172.21.15.79:0/883486675 shutdown_connections 2024-04-26T19:51:16.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.234+0000 7f3df44d1640 1 -- 172.21.15.79:0/883486675 wait complete. 2024-04-26T19:51:16.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.235+0000 7f3df44d1640 1 Processor -- start 2024-04-26T19:51:16.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.235+0000 7f3df44d1640 1 -- start start 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df44d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd80039b0 0x7f3dec06f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df44d1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dec141d50 0x7f3dec156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df44d1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec157380 0x7f3dec15be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df44d1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec06d190 con 0x7f3dec141d50 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df44d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3dec06d010 con 0x7f3dd80039b0 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df44d1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3dec0b9340 con 0x7f3dec157380 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df1a6c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dec141d50 0x7f3dec156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df1a6c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dec141d50 0x7f3dec156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44364/0 (socket says 172.21.15.79:44364) 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df1a6c640 1 -- 172.21.15.79:0/3020940547 learned_addr learned my addr 172.21.15.79:0/3020940547 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df226d640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd80039b0 0x7f3dec06f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df2a6e640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec157380 0x7f3dec15be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df1a6c640 1 -- 172.21.15.79:0/3020940547 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec157380 msgr2=0x7f3dec15be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df1a6c640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec157380 0x7f3dec15be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df1a6c640 1 -- 172.21.15.79:0/3020940547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd80039b0 msgr2=0x7f3dec06f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df1a6c640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd80039b0 0x7f3dec06f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df1a6c640 1 -- 172.21.15.79:0/3020940547 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dec15ce50 con 0x7f3dec141d50 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.236+0000 7f3df226d640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd80039b0 0x7f3dec06f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.237+0000 7f3df1a6c640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dec141d50 0x7f3dec156e40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b7f0 tx=0x7f3ddc00bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.237+0000 7f3de2ffd640 1 -- 172.21.15.79:0/3020940547 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc018070 con 0x7f3dec141d50 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.237+0000 7f3de2ffd640 1 -- 172.21.15.79:0/3020940547 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0107d0 con 0x7f3dec141d50 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.237+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dec06e5c0 con 0x7f3dec141d50 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.237+0000 7f3de2ffd640 1 -- 172.21.15.79:0/3020940547 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc010ba0 con 0x7f3dec141d50 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.237+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dec06e860 con 0x7f3dec141d50 2024-04-26T19:51:16.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.238+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dec0506c0 con 0x7f3dec141d50 2024-04-26T19:51:16.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.238+0000 7f3de2ffd640 1 -- 172.21.15.79:0/3020940547 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ddc01b070 con 0x7f3dec141d50 2024-04-26T19:51:16.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.239+0000 7f3de2ffd640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3dd006c8f0 0x7f3dd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.239+0000 7f3de2ffd640 1 -- 172.21.15.79:0/3020940547 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3ddc08c220 con 0x7f3dec141d50 2024-04-26T19:51:16.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.239+0000 7f3df226d640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3dd006c8f0 0x7f3dd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.239+0000 7f3df226d640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3dd006c8f0 0x7f3dd006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ba00 tx=0x7f3de8030000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:16.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.243+0000 7f3de2ffd640 1 -- 172.21.15.79:0/3020940547 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ddc05a2e0 con 0x7f3dec141d50 2024-04-26T19:51:16.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.419+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f3dec0b4080 con 0x7f3dec141d50 2024-04-26T19:51:16.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.420+0000 7f3de2ffd640 1 -- 172.21.15.79:0/3020940547 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f3ddc059be0 con 0x7f3dec141d50 2024-04-26T19:51:16.420 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3dd006c8f0 msgr2=0x7f3dd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3dd006c8f0 0x7f3dd006edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3de800ba00 tx=0x7f3de8030000 comp rx=0 tx=0).stop 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dec141d50 msgr2=0x7f3dec156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dec141d50 0x7f3dec156e40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc00b7f0 tx=0x7f3ddc00bcb0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 shutdown_connections 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3dd006c8f0 0x7f3dd006edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3dec157380 0x7f3dec15be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3dec141d50 0x7f3dec156e40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 --2- 172.21.15.79:0/3020940547 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3dd80039b0 0x7f3dec06f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 >> 172.21.15.79:0/3020940547 conn(0x7f3dec073b40 msgr2=0x7f3dec1598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 shutdown_connections 2024-04-26T19:51:16.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.422+0000 7f3df44d1640 1 -- 172.21.15.79:0/3020940547 wait complete. 2024-04-26T19:51:16.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:16.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.529+0000 7ff91d7c1640 1 Processor -- start 2024-04-26T19:51:16.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.529+0000 7ff91d7c1640 1 -- start start 2024-04-26T19:51:16.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff91d7c1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff91806fb20 0x7ff9180d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff91d7c1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9180b1290 con 0x7ff91806ff00 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff91d7c1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9180d4090 con 0x7ff918148fd0 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff91d7c1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff9180d4370 con 0x7ff91806fb20 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff9167fc640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff91806ff00 0x7ff9181488c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35324/0 (socket says 172.21.15.79:35324) 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff9167fc640 1 -- 172.21.15.79:0/2662227460 learned_addr learned my addr 172.21.15.79:0/2662227460 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff916ffd640 1 --2- 172.21.15.79:0/2662227460 >> v2:172.21.15.83:3300/0 conn(0x7ff91806fb20 0x7ff9180d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff915ffb640 1 -- 172.21.15.79:0/2662227460 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3014879024 0 0) 0x7ff9180b1290 con 0x7ff91806ff00 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff915ffb640 1 -- 172.21.15.79:0/2662227460 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff900003520 con 0x7ff91806ff00 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff916ffd640 1 -- 172.21.15.79:0/2662227460 >> v1:172.21.15.83:6789/0 conn(0x7ff918148fd0 legacy=0x7ff9180aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff915ffb640 1 -- 172.21.15.79:0/2662227460 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 284696276 0 0) 0x7ff900003520 con 0x7ff91806ff00 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff916ffd640 1 -- 172.21.15.79:0/2662227460 >> v1:172.21.15.79:6789/0 conn(0x7ff91806ff00 legacy=0x7ff9181488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.530+0000 7ff916ffd640 1 -- 172.21.15.79:0/2662227460 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9180d4650 con 0x7ff91806fb20 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.531+0000 7ff916ffd640 1 --2- 172.21.15.79:0/2662227460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806fb20 0x7ff9180d2cd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff908009800 tx=0x7ff90802a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f124af297a4c73d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:16.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.531+0000 7ff915ffb640 1 -- 172.21.15.79:0/2662227460 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908037070 con 0x7ff91806fb20 2024-04-26T19:51:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.531+0000 7ff915ffb640 1 -- 172.21.15.79:0/2662227460 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90802ad30 con 0x7ff91806fb20 2024-04-26T19:51:16.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.531+0000 7ff915ffb640 1 -- 172.21.15.79:0/2662227460 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90c001670 con 0x7ff91806fb20 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.532+0000 7ff91d7c1640 1 -- 172.21.15.79:0/2662227460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806fb20 msgr2=0x7ff9180d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.532+0000 7ff91d7c1640 1 --2- 172.21.15.79:0/2662227460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806fb20 0x7ff9180d2cd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff908009800 tx=0x7ff90802a2a0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.532+0000 7ff91d7c1640 1 -- 172.21.15.79:0/2662227460 shutdown_connections 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.532+0000 7ff91d7c1640 1 --2- 172.21.15.79:0/2662227460 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806fb20 0x7ff9180d2cd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.532+0000 7ff91d7c1640 1 -- 172.21.15.79:0/2662227460 >> 172.21.15.79:0/2662227460 conn(0x7ff918071c30 msgr2=0x7ff91806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.532+0000 7ff91d7c1640 1 -- 172.21.15.79:0/2662227460 shutdown_connections 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.532+0000 7ff91d7c1640 1 -- 172.21.15.79:0/2662227460 wait complete. 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.533+0000 7ff91d7c1640 1 Processor -- start 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.533+0000 7ff91d7c1640 1 -- start start 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff91d7c1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff91806fb20 0x7ff918146310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff91d7c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806ff00 0x7ff918147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff91d7c1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff918148fd0 0x7ff918161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff91d7c1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff9180d3920 con 0x7ff91806fb20 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff91d7c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff9180d4c10 con 0x7ff91806ff00 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff91d7c1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7ff9180d3aa0 con 0x7ff918148fd0 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff916ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff91806fb20 0x7ff918146310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff916ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff91806fb20 0x7ff918146310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44366/0 (socket says 172.21.15.79:44366) 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff916ffd640 1 -- 172.21.15.79:0/244194049 learned_addr learned my addr 172.21.15.79:0/244194049 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff9167fc640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806ff00 0x7ff918147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff916ffd640 1 -- 172.21.15.79:0/244194049 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff918148fd0 msgr2=0x7ff918161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff9177fe640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff918148fd0 0x7ff918161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff916ffd640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff918148fd0 0x7ff918161fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff916ffd640 1 -- 172.21.15.79:0/244194049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806ff00 msgr2=0x7ff918147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff916ffd640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806ff00 0x7ff918147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff916ffd640 1 -- 172.21.15.79:0/244194049 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff918146850 con 0x7ff91806fb20 2024-04-26T19:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff9167fc640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806ff00 0x7ff918147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff916ffd640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff91806fb20 0x7ff918146310 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff908009920 tx=0x7ff9080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff8ff7fe640 1 -- 172.21.15.79:0/244194049 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908042020 con 0x7ff91806fb20 2024-04-26T19:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff8ff7fe640 1 -- 172.21.15.79:0/244194049 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff90803d670 con 0x7ff91806fb20 2024-04-26T19:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff918146af0 con 0x7ff91806fb20 2024-04-26T19:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff8ff7fe640 1 -- 172.21.15.79:0/244194049 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff908037040 con 0x7ff91806fb20 2024-04-26T19:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.534+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9181470b0 con 0x7ff91806fb20 2024-04-26T19:51:16.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.535+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9180506c0 con 0x7ff91806fb20 2024-04-26T19:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.536+0000 7ff8ff7fe640 1 -- 172.21.15.79:0/244194049 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff908045020 con 0x7ff91806fb20 2024-04-26T19:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.536+0000 7ff8ff7fe640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff90406c8f0 0x7ff90406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.536+0000 7ff9167fc640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff90406c8f0 0x7ff90406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.536+0000 7ff8ff7fe640 1 -- 172.21.15.79:0/244194049 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff9080ac440 con 0x7ff91806fb20 2024-04-26T19:51:16.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.537+0000 7ff9167fc640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff90406c8f0 0x7ff90406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff918145e30 tx=0x7ff90c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:16.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.541+0000 7ff8ff7fe640 1 -- 172.21.15.79:0/244194049 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff90804c070 con 0x7ff91806fb20 2024-04-26T19:51:16.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.715+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9181480d0 con 0x7ff91806fb20 2024-04-26T19:51:16.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.717+0000 7ff8ff7fe640 1 -- 172.21.15.79:0/244194049 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7ff90807a500 con 0x7ff91806fb20 2024-04-26T19:51:16.717 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:16.718 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:16.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.718+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff90406c8f0 msgr2=0x7ff90406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.718+0000 7ff91d7c1640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff90406c8f0 0x7ff90406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff918145e30 tx=0x7ff90c009040 comp rx=0 tx=0).stop 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.718+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff91806fb20 msgr2=0x7ff918146310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.718+0000 7ff91d7c1640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff91806fb20 0x7ff918146310 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff908009920 tx=0x7ff9080046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.719+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 shutdown_connections 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.719+0000 7ff91d7c1640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff90406c8f0 0x7ff90406edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.719+0000 7ff91d7c1640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff918148fd0 0x7ff918161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.719+0000 7ff91d7c1640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff91806ff00 0x7ff918147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.719+0000 7ff91d7c1640 1 --2- 172.21.15.79:0/244194049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff91806fb20 0x7ff918146310 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.719+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 >> 172.21.15.79:0/244194049 conn(0x7ff918071c30 msgr2=0x7ff918072a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.719+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 shutdown_connections 2024-04-26T19:51:16.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.719+0000 7ff91d7c1640 1 -- 172.21.15.79:0/244194049 wait complete. 2024-04-26T19:51:16.730 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:16.731 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:16.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.829+0000 7f9903b78640 1 Processor -- start 2024-04-26T19:51:16.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.830+0000 7f9903b78640 1 -- start start 2024-04-26T19:51:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.830+0000 7f9903b78640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc15c1e0 0x7f98fc073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.830+0000 7f9903b78640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98fc072620 con 0x7f98fc06f920 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9903b78640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98fc0730d0 con 0x7f98fc072000 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9903b78640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f98fc0733b0 con 0x7f98fc15c1e0 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9901113640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f98fc06f920 0x7f98fc0718f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35332/0 (socket says 172.21.15.79:35332) 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9901113640 1 -- 172.21.15.79:0/313906244 learned_addr learned my addr 172.21.15.79:0/313906244 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9901914640 1 --2- 172.21.15.79:0/313906244 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc15c1e0 0x7f98fc073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9900912640 1 -- 172.21.15.79:0/313906244 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4044298641 0 0) 0x7f98fc072620 con 0x7f98fc06f920 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9900912640 1 -- 172.21.15.79:0/313906244 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98e8005730 con 0x7f98fc06f920 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9900912640 1 -- 172.21.15.79:0/313906244 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 663615462 0 0) 0x7f98fc0730d0 con 0x7f98fc072000 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9900912640 1 -- 172.21.15.79:0/313906244 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f98fc072620 con 0x7f98fc072000 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9901914640 1 -- 172.21.15.79:0/313906244 >> v1:172.21.15.83:6789/0 conn(0x7f98fc072000 legacy=0x7f98fc0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9900912640 1 -- 172.21.15.79:0/313906244 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2588022746 0 0) 0x7f98e8005730 con 0x7f98fc06f920 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9901914640 1 -- 172.21.15.79:0/313906244 >> v1:172.21.15.79:6789/0 conn(0x7f98fc06f920 legacy=0x7f98fc0718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.831+0000 7f9901914640 1 -- 172.21.15.79:0/313906244 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc073690 con 0x7f98fc15c1e0 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.832+0000 7f9901914640 1 --2- 172.21.15.79:0/313906244 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc15c1e0 0x7f98fc073bb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f98f00097d0 tx=0x7f98f002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6317bbc549f3c3d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:16.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.832+0000 7f9900912640 1 -- 172.21.15.79:0/313906244 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0037070 con 0x7f98fc15c1e0 2024-04-26T19:51:16.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.832+0000 7f9900912640 1 -- 172.21.15.79:0/313906244 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f002ab10 con 0x7f98fc15c1e0 2024-04-26T19:51:16.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.832+0000 7f9900912640 1 -- 172.21.15.79:0/313906244 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f8001670 con 0x7f98fc15c1e0 2024-04-26T19:51:16.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.833+0000 7f9903b78640 1 -- 172.21.15.79:0/313906244 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc15c1e0 msgr2=0x7f98fc073bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.833+0000 7f9903b78640 1 --2- 172.21.15.79:0/313906244 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc15c1e0 0x7f98fc073bb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f98f00097d0 tx=0x7f98f002a080 comp rx=0 tx=0).stop 2024-04-26T19:51:16.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.833+0000 7f9903b78640 1 -- 172.21.15.79:0/313906244 shutdown_connections 2024-04-26T19:51:16.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.833+0000 7f9903b78640 1 --2- 172.21.15.79:0/313906244 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc15c1e0 0x7f98fc073bb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.833+0000 7f9903b78640 1 -- 172.21.15.79:0/313906244 >> 172.21.15.79:0/313906244 conn(0x7f98fc0b2540 msgr2=0x7f98fc0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.833+0000 7f9903b78640 1 -- 172.21.15.79:0/313906244 shutdown_connections 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.833+0000 7f9903b78640 1 -- 172.21.15.79:0/313906244 wait complete. 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.834+0000 7f9903b78640 1 Processor -- start 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.834+0000 7f9903b78640 1 -- start start 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.834+0000 7f9903b78640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98fc06f920 0x7f98fc148cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9903b78640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc072000 0x7f98fc158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9903b78640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98fc15c1e0 0x7f98fc0ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9903b78640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc070190 con 0x7f98fc06f920 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9903b78640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc070010 con 0x7f98fc15c1e0 2024-04-26T19:51:16.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9903b78640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f98fc0ce3a0 con 0x7f98fc072000 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9901914640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98fc06f920 0x7f98fc148cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9901914640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98fc06f920 0x7f98fc148cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44382/0 (socket says 172.21.15.79:44382) 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9901914640 1 -- 172.21.15.79:0/4039518525 learned_addr learned my addr 172.21.15.79:0/4039518525 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9901914640 1 -- 172.21.15.79:0/4039518525 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc072000 msgr2=0x7f98fc158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9901914640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc072000 0x7f98fc158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9901914640 1 -- 172.21.15.79:0/4039518525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98fc15c1e0 msgr2=0x7f98fc0ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9901914640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98fc15c1e0 0x7f98fc0ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9901914640 1 -- 172.21.15.79:0/4039518525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc1f5cd0 con 0x7f98fc06f920 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9901914640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98fc06f920 0x7f98fc148cf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f98f00097d0 tx=0x7f98f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f98ee7fc640 1 -- 172.21.15.79:0/4039518525 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f0037050 con 0x7f98fc06f920 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f98ee7fc640 1 -- 172.21.15.79:0/4039518525 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98f0043070 con 0x7f98fc06f920 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc1f5fb0 con 0x7f98fc06f920 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f98ee7fc640 1 -- 172.21.15.79:0/4039518525 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f003ca50 con 0x7f98fc06f920 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.835+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc1f6570 con 0x7f98fc06f920 2024-04-26T19:51:16.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.836+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98fc0506c0 con 0x7f98fc06f920 2024-04-26T19:51:16.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.837+0000 7f98ee7fc640 1 -- 172.21.15.79:0/4039518525 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98f003cbf0 con 0x7f98fc06f920 2024-04-26T19:51:16.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.837+0000 7f98ee7fc640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f98e006c8f0 0x7f98e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:16.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.837+0000 7f98ee7fc640 1 -- 172.21.15.79:0/4039518525 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f98f00aca00 con 0x7f98fc06f920 2024-04-26T19:51:16.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.837+0000 7f9901113640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f98e006c8f0 0x7f98e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:16.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.838+0000 7f9901113640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f98e006c8f0 0x7f98e006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0cd280 tx=0x7f98f8000f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:16.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:16.841+0000 7f98ee7fc640 1 -- 172.21.15.79:0/4039518525 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f98f0046360 con 0x7f98fc06f920 2024-04-26T19:51:17.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.014+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f98fc0b22b0 con 0x7f98fc06f920 2024-04-26T19:51:17.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.014+0000 7f98ee7fc640 1 -- 172.21.15.79:0/4039518525 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f98f007aa30 con 0x7f98fc06f920 2024-04-26T19:51:17.015 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:17.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f98e006c8f0 msgr2=0x7f98e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f98e006c8f0 0x7f98e006edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0cd280 tx=0x7f98f8000f90 comp rx=0 tx=0).stop 2024-04-26T19:51:17.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98fc06f920 msgr2=0x7f98fc148cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98fc06f920 0x7f98fc148cf0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f98f00097d0 tx=0x7f98f00046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 shutdown_connections 2024-04-26T19:51:17.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f98e006c8f0 0x7f98e006edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f98fc15c1e0 0x7f98fc0ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f98fc072000 0x7f98fc158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 --2- 172.21.15.79:0/4039518525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f98fc06f920 0x7f98fc148cf0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 >> 172.21.15.79:0/4039518525 conn(0x7f98fc0b2540 msgr2=0x7f98fc0d0600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:17.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 shutdown_connections 2024-04-26T19:51:17.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.016+0000 7f9903b78640 1 -- 172.21.15.79:0/4039518525 wait complete. 2024-04-26T19:51:17.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.126+0000 7f95638d9640 1 Processor -- start 2024-04-26T19:51:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.126+0000 7f95638d9640 1 -- start start 2024-04-26T19:51:17.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f95638d9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f955c0d1830 0x7f955c1615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f95638d9640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f955c0d1dd0 con 0x7f955c06ca60 2024-04-26T19:51:17.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f95638d9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f955c0d20b0 con 0x7f955c0b8f60 2024-04-26T19:51:17.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f95638d9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f955c0d2950 con 0x7f955c0d1830 2024-04-26T19:51:17.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9560e74640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f955c06ca60 0x7f955c160ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:35334/0 (socket says 172.21.15.79:35334) 2024-04-26T19:51:17.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9560e74640 1 -- 172.21.15.79:0/1063898862 learned_addr learned my addr 172.21.15.79:0/1063898862 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:17.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9561e76640 1 --2- 172.21.15.79:0/1063898862 >> v2:172.21.15.83:3300/0 conn(0x7f955c0d1830 0x7f955c1615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840666622 0 0) 0x7f955c0d1dd0 con 0x7f955c06ca60 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9548003520 con 0x7f955c06ca60 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2677808003 0 0) 0x7f9548003520 con 0x7f955c06ca60 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 >> v2:172.21.15.83:3300/0 conn(0x7f955c0d1830 msgr2=0x7f955c1615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 --2- 172.21.15.79:0/1063898862 >> v2:172.21.15.83:3300/0 conn(0x7f955c0d1830 0x7f955c1615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 >> v1:172.21.15.83:6789/0 conn(0x7f955c0b8f60 legacy=0x7f955c06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c0d2390 con 0x7f955c06ca60 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f954c001670 con 0x7f955c06ca60 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 >> v1:172.21.15.79:6789/0 conn(0x7f955c06ca60 legacy=0x7f955c160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95480039b0 0x7f9548003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9548004800 0x7f9548004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480051c0 0x7f954800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f955c0d2950 con 0x7f95480039b0 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f954800c070 con 0x7f95480051c0 2024-04-26T19:51:17.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f954800c270 con 0x7f9548004800 2024-04-26T19:51:17.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9561e76640 1 --2- 172.21.15.79:0/1063898862 >> v2:172.21.15.83:3300/0 conn(0x7f955c0d1830 0x7f955c1615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:17.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.127+0000 7f9561675640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95480039b0 0x7f9548003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.128+0000 7f9560e74640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9548004800 0x7f9548004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.128+0000 7f9561e76640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480051c0 0x7f954800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.128+0000 7f9561675640 1 -- 172.21.15.79:0/1063898862 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9548004800 msgr2=0x7f9548004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:17.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.128+0000 7f9561675640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9548004800 0x7f9548004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.128+0000 7f9561675640 1 -- 172.21.15.79:0/1063898862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480051c0 msgr2=0x7f954800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:17.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.128+0000 7f9561675640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f95480051c0 0x7f954800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.128+0000 7f9561e76640 1 -- 172.21.15.79:0/1063898862 reap_dead start 2024-04-26T19:51:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.128+0000 7f9561675640 1 -- 172.21.15.79:0/1063898862 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9548003520 con 0x7f95480039b0 2024-04-26T19:51:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.128+0000 7f9561675640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95480039b0 0x7f9548003dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9558001750 tx=0x7f9548005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd27b4473ec619a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558001370 con 0x7f95480039b0 2024-04-26T19:51:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95580052d0 con 0x7f95480039b0 2024-04-26T19:51:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f9553fff640 1 -- 172.21.15.79:0/1063898862 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955802abc0 con 0x7f95480039b0 2024-04-26T19:51:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f95638d9640 1 -- 172.21.15.79:0/1063898862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95480039b0 msgr2=0x7f9548003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f95638d9640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95480039b0 0x7f9548003dd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9558001750 tx=0x7f9548005830 comp rx=0 tx=0).stop 2024-04-26T19:51:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f95638d9640 1 -- 172.21.15.79:0/1063898862 shutdown_connections 2024-04-26T19:51:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f95638d9640 1 --2- 172.21.15.79:0/1063898862 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f95480039b0 0x7f9548003dd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f95638d9640 1 -- 172.21.15.79:0/1063898862 >> 172.21.15.79:0/1063898862 conn(0x7f955c073b40 msgr2=0x7f955c073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f95638d9640 1 -- 172.21.15.79:0/1063898862 shutdown_connections 2024-04-26T19:51:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.129+0000 7f95638d9640 1 -- 172.21.15.79:0/1063898862 wait complete. 2024-04-26T19:51:17.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.130+0000 7f95638d9640 1 Processor -- start 2024-04-26T19:51:17.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.130+0000 7f95638d9640 1 -- start start 2024-04-26T19:51:17.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f95638d9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95480039b0 0x7f955c154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f95638d9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f955c1533d0 0x7f955c15c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f95638d9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f955c1e9250 0x7f955c06e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f95638d9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f955c06d190 con 0x7f955c1533d0 2024-04-26T19:51:17.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f95638d9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f955c06d010 con 0x7f955c1e9250 2024-04-26T19:51:17.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f95638d9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f955c0b9340 con 0x7f95480039b0 2024-04-26T19:51:17.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9560e74640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f955c1533d0 0x7f955c15c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9560e74640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f955c1533d0 0x7f955c15c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44398/0 (socket says 172.21.15.79:44398) 2024-04-26T19:51:17.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9560e74640 1 -- 172.21.15.79:0/2541272831 learned_addr learned my addr 172.21.15.79:0/2541272831 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:17.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9561675640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95480039b0 0x7f955c154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9561e76640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f955c1e9250 0x7f955c06e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9560e74640 1 -- 172.21.15.79:0/2541272831 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95480039b0 msgr2=0x7f955c154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9560e74640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95480039b0 0x7f955c154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9560e74640 1 -- 172.21.15.79:0/2541272831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f955c1e9250 msgr2=0x7f955c06e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9560e74640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f955c1e9250 0x7f955c06e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9560e74640 1 -- 172.21.15.79:0/2541272831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955c154e40 con 0x7f955c1533d0 2024-04-26T19:51:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.131+0000 7f9561e76640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f955c1e9250 0x7f955c06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.132+0000 7f9560e74640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f955c1533d0 0x7f955c15c7b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f954c004ed0 tx=0x7f954c007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.132+0000 7f9551ffb640 1 -- 172.21.15.79:0/2541272831 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c00f7b0 con 0x7f955c1533d0 2024-04-26T19:51:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.132+0000 7f9551ffb640 1 -- 172.21.15.79:0/2541272831 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f954c018070 con 0x7f955c1533d0 2024-04-26T19:51:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.132+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955c153860 con 0x7f955c1533d0 2024-04-26T19:51:17.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.132+0000 7f9551ffb640 1 -- 172.21.15.79:0/2541272831 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c00fbb0 con 0x7f955c1533d0 2024-04-26T19:51:17.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.132+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955c153e20 con 0x7f955c1533d0 2024-04-26T19:51:17.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.133+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955c1545e0 con 0x7f955c1533d0 2024-04-26T19:51:17.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.133+0000 7f9551ffb640 1 -- 172.21.15.79:0/2541272831 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f954c00fd50 con 0x7f955c1533d0 2024-04-26T19:51:17.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.134+0000 7f9551ffb640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f954006c8f0 0x7f954006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.134+0000 7f9551ffb640 1 -- 172.21.15.79:0/2541272831 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f954c08be40 con 0x7f955c1533d0 2024-04-26T19:51:17.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.134+0000 7f9561675640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f954006c8f0 0x7f954006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.134+0000 7f9561675640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f954006c8f0 0x7f954006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9558032000 tx=0x7f955802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:17.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.138+0000 7f9551ffb640 1 -- 172.21.15.79:0/2541272831 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f955c1545e0 con 0x7f955c1533d0 2024-04-26T19:51:17.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.316+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f955c0b4080 con 0x7f955c1533d0 2024-04-26T19:51:17.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.317+0000 7f9551ffb640 1 -- 172.21.15.79:0/2541272831 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f955c154e40 con 0x7f955c1533d0 2024-04-26T19:51:17.318 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f954006c8f0 msgr2=0x7f954006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f954006c8f0 0x7f954006edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9558032000 tx=0x7f955802e000 comp rx=0 tx=0).stop 2024-04-26T19:51:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f955c1533d0 msgr2=0x7f955c15c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f955c1533d0 0x7f955c15c7b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f954c004ed0 tx=0x7f954c007430 comp rx=0 tx=0).stop 2024-04-26T19:51:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 shutdown_connections 2024-04-26T19:51:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f954006c8f0 0x7f954006edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f955c1e9250 0x7f955c06e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f955c1533d0 0x7f955c15c7b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 --2- 172.21.15.79:0/2541272831 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f95480039b0 0x7f955c154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 >> 172.21.15.79:0/2541272831 conn(0x7f955c073b40 msgr2=0x7f955c162bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:17.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 shutdown_connections 2024-04-26T19:51:17.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.319+0000 7f95638d9640 1 -- 172.21.15.79:0/2541272831 wait complete. 2024-04-26T19:51:17.330 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:17.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.428+0000 7f3357fac640 1 Processor -- start 2024-04-26T19:51:17.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.429+0000 7f3357fac640 1 -- start start 2024-04-26T19:51:17.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.429+0000 7f3357fac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3350142480 0x7f3350163010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.429+0000 7f3357fac640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3350160050 0x7f3350160460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.429+0000 7f3357fac640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3350163e90 con 0x7f3350071f90 2024-04-26T19:51:17.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.429+0000 7f3357fac640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f33501635f0 con 0x7f3350160050 2024-04-26T19:51:17.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.429+0000 7f3357fac640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f33501638d0 con 0x7f3350142480 2024-04-26T19:51:17.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.429+0000 7f3356549640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3350160050 0x7f3350160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.430+0000 7f3356549640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3350160050 0x7f3350160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44406/0 (socket says 172.21.15.79:44406) 2024-04-26T19:51:17.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.430+0000 7f3356549640 1 -- 172.21.15.79:0/2968288182 learned_addr learned my addr 172.21.15.79:0/2968288182 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.430+0000 7f3355d48640 1 --2- 172.21.15.79:0/2968288182 >> v2:172.21.15.83:3300/0 conn(0x7f3350142480 0x7f3350163010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.430+0000 7f3356549640 1 -- 172.21.15.79:0/2968288182 >> v2:172.21.15.83:3300/0 conn(0x7f3350142480 msgr2=0x7f3350163010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.430+0000 7f3356549640 1 --2- 172.21.15.79:0/2968288182 >> v2:172.21.15.83:3300/0 conn(0x7f3350142480 0x7f3350163010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.430+0000 7f3356549640 1 -- 172.21.15.79:0/2968288182 >> v1:172.21.15.83:6789/0 conn(0x7f3350071f90 legacy=0x7f3350144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.430+0000 7f3356549640 1 -- 172.21.15.79:0/2968288182 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3350163bb0 con 0x7f3350160050 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.430+0000 7f3355d48640 1 --2- 172.21.15.79:0/2968288182 >> v2:172.21.15.83:3300/0 conn(0x7f3350142480 0x7f3350163010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.430+0000 7f3356549640 1 --2- 172.21.15.79:0/2968288182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350160050 0x7f3350160460 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3350072780 tx=0x7f334802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4537962d7428c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3354d46640 1 -- 172.21.15.79:0/2968288182 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3348036070 con 0x7f3350160050 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3354d46640 1 -- 172.21.15.79:0/2968288182 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3348031600 con 0x7f3350160050 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3354d46640 1 -- 172.21.15.79:0/2968288182 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33480319d0 con 0x7f3350160050 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3357fac640 1 -- 172.21.15.79:0/2968288182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350160050 msgr2=0x7f3350160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.434 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3357fac640 1 --2- 172.21.15.79:0/2968288182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350160050 0x7f3350160460 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3350072780 tx=0x7f334802a6f0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3357fac640 1 -- 172.21.15.79:0/2968288182 shutdown_connections 2024-04-26T19:51:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3357fac640 1 --2- 172.21.15.79:0/2968288182 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350160050 0x7f3350160460 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3357fac640 1 --2- 172.21.15.79:0/2968288182 >> v2:172.21.15.83:3300/0 conn(0x7f3350142480 0x7f3350163010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3357fac640 1 -- 172.21.15.79:0/2968288182 >> 172.21.15.79:0/2968288182 conn(0x7f33500b3ed0 msgr2=0x7f33500681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3357fac640 1 -- 172.21.15.79:0/2968288182 shutdown_connections 2024-04-26T19:51:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.431+0000 7f3357fac640 1 -- 172.21.15.79:0/2968288182 wait complete. 2024-04-26T19:51:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.432+0000 7f3357fac640 1 Processor -- start 2024-04-26T19:51:17.435 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.432+0000 7f3357fac640 1 -- start start 2024-04-26T19:51:17.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3357fac640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350071f90 0x7f3350155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3357fac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350142480 0x7f3350159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3357fac640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3350160050 0x7f33501595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3357fac640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3350145270 con 0x7f3350071f90 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3357fac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f33500d1610 con 0x7f3350142480 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3357fac640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3350157a00 con 0x7f3350160050 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3355d48640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350071f90 0x7f3350155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3355d48640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350071f90 0x7f3350155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44420/0 (socket says 172.21.15.79:44420) 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3355d48640 1 -- 172.21.15.79:0/3384051118 learned_addr learned my addr 172.21.15.79:0/3384051118 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3355547640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350142480 0x7f3350159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3356549640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3350160050 0x7f33501595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3355d48640 1 -- 172.21.15.79:0/3384051118 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3350160050 msgr2=0x7f33501595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3355d48640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3350160050 0x7f33501595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3355d48640 1 -- 172.21.15.79:0/3384051118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350142480 msgr2=0x7f3350159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3355d48640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350142480 0x7f3350159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.433+0000 7f3355d48640 1 -- 172.21.15.79:0/3384051118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3350157ce0 con 0x7f3350071f90 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.434+0000 7f3355d48640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350071f90 0x7f3350155850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c820 tx=0x7f334c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.434+0000 7f33467fc640 1 -- 172.21.15.79:0/3384051118 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c015070 con 0x7f3350071f90 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.434+0000 7f33467fc640 1 -- 172.21.15.79:0/3384051118 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f334c007a30 con 0x7f3350071f90 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.434+0000 7f33467fc640 1 -- 172.21.15.79:0/3384051118 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334c007e00 con 0x7f3350071f90 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.434+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3350157fc0 con 0x7f3350071f90 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.434+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3350155d90 con 0x7f3350071f90 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.435+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33500506c0 con 0x7f3350071f90 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.435+0000 7f33467fc640 1 -- 172.21.15.79:0/3384051118 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f334c018070 con 0x7f3350071f90 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.436+0000 7f33467fc640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f333406c8f0 0x7f333406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.436+0000 7f33467fc640 1 -- 172.21.15.79:0/3384051118 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f334c08bef0 con 0x7f3350071f90 2024-04-26T19:51:17.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.436+0000 7f3355547640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f333406c8f0 0x7f333406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.436+0000 7f3355547640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f333406c8f0 0x7f333406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f33500b5cc0 tx=0x7f3340009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:17.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.440+0000 7f33467fc640 1 -- 172.21.15.79:0/3384051118 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f334c059f20 con 0x7f3350071f90 2024-04-26T19:51:17.616 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.615+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3350162d80 con 0x7f3350071f90 2024-04-26T19:51:17.617 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.616+0000 7f33467fc640 1 -- 172.21.15.79:0/3384051118 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f334c059820 con 0x7f3350071f90 2024-04-26T19:51:17.617 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:17.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f333406c8f0 msgr2=0x7f333406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f333406c8f0 0x7f333406edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f33500b5cc0 tx=0x7f3340009040 comp rx=0 tx=0).stop 2024-04-26T19:51:17.619 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350071f90 msgr2=0x7f3350155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350071f90 0x7f3350155850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c820 tx=0x7f334c00cc20 comp rx=0 tx=0).stop 2024-04-26T19:51:17.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 shutdown_connections 2024-04-26T19:51:17.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f333406c8f0 0x7f333406edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3350160050 0x7f33501595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3350142480 0x7f3350159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 --2- 172.21.15.79:0/3384051118 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3350071f90 0x7f3350155850 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 >> 172.21.15.79:0/3384051118 conn(0x7f33500b3ed0 msgr2=0x7f3350144870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:17.620 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 shutdown_connections 2024-04-26T19:51:17.621 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.618+0000 7f3357fac640 1 -- 172.21.15.79:0/3384051118 wait complete. 2024-04-26T19:51:17.629 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:17.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.729+0000 7fe4dc46d640 1 Processor -- start 2024-04-26T19:51:17.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.729+0000 7fe4dc46d640 1 -- start start 2024-04-26T19:51:17.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4dc46d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe4d4141d10 0x7fe4d40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4dc46d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d40af8b0 0x7fe4d40afd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4dc46d640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d40cfdd0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4dc46d640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d40b0250 con 0x7fe4d4141d10 2024-04-26T19:51:17.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4dc46d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d40d00b0 con 0x7fe4d40af8b0 2024-04-26T19:51:17.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4da209640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe4d4141d10 0x7fe4d40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4daa0a640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe4d415f2e0 0x7fe4d415f740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33220/0 (socket says 172.21.15.79:33220) 2024-04-26T19:51:17.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4daa0a640 1 -- 172.21.15.79:0/3030432372 learned_addr learned my addr 172.21.15.79:0/3030432372 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:17.743 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4d9a08640 1 --2- 172.21.15.79:0/3030432372 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d40af8b0 0x7fe4d40afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4da209640 1 -- 172.21.15.79:0/3030432372 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d40af8b0 msgr2=0x7fe4d40afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4da209640 1 --2- 172.21.15.79:0/3030432372 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d40af8b0 0x7fe4d40afd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4da209640 1 -- 172.21.15.79:0/3030432372 >> v1:172.21.15.79:6789/0 conn(0x7fe4d415f2e0 legacy=0x7fe4d415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4da209640 1 -- 172.21.15.79:0/3030432372 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d40d0390 con 0x7fe4d4141d10 2024-04-26T19:51:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.730+0000 7fe4d9207640 1 -- 172.21.15.79:0/3030432372 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1615840628 0 0) 0x7fe4d40cfdd0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.731+0000 7fe4da209640 1 --2- 172.21.15.79:0/3030432372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d4141d10 0x7fe4d40ce2a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009d90 tx=0x7fe4d002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc74b707b369d568 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.731+0000 7fe4d9207640 1 -- 172.21.15.79:0/3030432372 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0007740 con 0x7fe4d4141d10 2024-04-26T19:51:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.731+0000 7fe4d9207640 1 -- 172.21.15.79:0/3030432372 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d00078e0 con 0x7fe4d4141d10 2024-04-26T19:51:17.744 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.731+0000 7fe4d9207640 1 -- 172.21.15.79:0/3030432372 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d0007cb0 con 0x7fe4d4141d10 2024-04-26T19:51:17.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.732+0000 7fe4dc46d640 1 -- 172.21.15.79:0/3030432372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d4141d10 msgr2=0x7fe4d40ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.732+0000 7fe4dc46d640 1 --2- 172.21.15.79:0/3030432372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d4141d10 0x7fe4d40ce2a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009d90 tx=0x7fe4d002a660 comp rx=0 tx=0).stop 2024-04-26T19:51:17.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.732+0000 7fe4dc46d640 1 -- 172.21.15.79:0/3030432372 shutdown_connections 2024-04-26T19:51:17.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.732+0000 7fe4dc46d640 1 --2- 172.21.15.79:0/3030432372 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d40af8b0 0x7fe4d40afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.732+0000 7fe4dc46d640 1 --2- 172.21.15.79:0/3030432372 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d4141d10 0x7fe4d40ce2a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.745 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.732+0000 7fe4dc46d640 1 -- 172.21.15.79:0/3030432372 >> 172.21.15.79:0/3030432372 conn(0x7fe4d40b2540 msgr2=0x7fe4d40b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:17.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.732+0000 7fe4dc46d640 1 -- 172.21.15.79:0/3030432372 shutdown_connections 2024-04-26T19:51:17.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.732+0000 7fe4dc46d640 1 -- 172.21.15.79:0/3030432372 wait complete. 2024-04-26T19:51:17.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.733+0000 7fe4dc46d640 1 Processor -- start 2024-04-26T19:51:17.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.733+0000 7fe4dc46d640 1 -- start start 2024-04-26T19:51:17.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4dc46d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d40af8b0 0x7fe4d41349a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4dc46d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d4141d10 0x7fe4d4154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4dc46d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d415f2e0 0x7fe4d40ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4dc46d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40d0c30 con 0x7fe4d415f2e0 2024-04-26T19:51:17.746 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4dc46d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d40d0950 con 0x7fe4d40af8b0 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4dc46d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d40ce7e0 con 0x7fe4d4141d10 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4daa0a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d415f2e0 0x7fe4d40ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4daa0a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d415f2e0 0x7fe4d40ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56944/0 (socket says 172.21.15.79:56944) 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4daa0a640 1 -- 172.21.15.79:0/647867605 learned_addr learned my addr 172.21.15.79:0/647867605 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4d9a08640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d4141d10 0x7fe4d4154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4da209640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d40af8b0 0x7fe4d41349a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4daa0a640 1 -- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d4141d10 msgr2=0x7fe4d4154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4daa0a640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d4141d10 0x7fe4d4154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4daa0a640 1 -- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d40af8b0 msgr2=0x7fe4d41349a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.747 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4daa0a640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d40af8b0 0x7fe4d41349a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4daa0a640 1 -- 172.21.15.79:0/647867605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d4134ee0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4da209640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d40af8b0 0x7fe4d41349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4d9a08640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d4141d10 0x7fe4d4154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4daa0a640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d415f2e0 0x7fe4d40ae8d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c440 tx=0x7fe4cc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4caffd640 1 -- 172.21.15.79:0/647867605 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc0074e0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4caffd640 1 -- 172.21.15.79:0/647867605 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4cc0047e0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d4136af0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4caffd640 1 -- 172.21.15.79:0/647867605 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc004bd0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.734+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d4169d00 con 0x7fe4d415f2e0 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.735+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d416a7c0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.736+0000 7fe4caffd640 1 -- 172.21.15.79:0/647867605 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4cc011070 con 0x7fe4d415f2e0 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.736+0000 7fe4caffd640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe4b806c8f0 0x7fe4b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:17.748 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.737+0000 7fe4da209640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe4b806c8f0 0x7fe4b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:17.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.737+0000 7fe4caffd640 1 -- 172.21.15.79:0/647867605 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe4cc08baa0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.737+0000 7fe4da209640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe4b806c8f0 0x7fe4b806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0034000 tx=0x7fe4d002f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:17.749 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.741+0000 7fe4caffd640 1 -- 172.21.15.79:0/647867605 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4d416a7c0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.915+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe4d40506c0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.915+0000 7fe4caffd640 1 -- 172.21.15.79:0/647867605 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fe4cc059ad0 con 0x7fe4d415f2e0 2024-04-26T19:51:17.916 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:17.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe4b806c8f0 msgr2=0x7fe4b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe4b806c8f0 0x7fe4b806edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0034000 tx=0x7fe4d002f000 comp rx=0 tx=0).stop 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d415f2e0 msgr2=0x7fe4d40ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d415f2e0 0x7fe4d40ae8d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c440 tx=0x7fe4cc00c900 comp rx=0 tx=0).stop 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 shutdown_connections 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe4b806c8f0 0x7fe4b806edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4d415f2e0 0x7fe4d40ae8d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4d4141d10 0x7fe4d4154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 --2- 172.21.15.79:0/647867605 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4d40af8b0 0x7fe4d41349a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 >> 172.21.15.79:0/647867605 conn(0x7fe4d40b2540 msgr2=0x7fe4d40b1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 shutdown_connections 2024-04-26T19:51:17.918 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:17.917+0000 7fe4dc46d640 1 -- 172.21.15.79:0/647867605 wait complete. 2024-04-26T19:51:17.947 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2024-04-26T19:51:17.947 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2024-04-26T19:51:17.948 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:18.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.044+0000 7fc1ff178640 1 Processor -- start 2024-04-26T19:51:18.044 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.044+0000 7fc1ff178640 1 -- start start 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.044+0000 7fc1ff178640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f80aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.044+0000 7fc1ff178640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1f80aaa00 0x7fc1f8160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.044+0000 7fc1ff178640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1f80d15d0 con 0x7fc1f80b3d80 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.044+0000 7fc1ff178640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc1f80d2f80 con 0x7fc1f80aaa00 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.044+0000 7fc1ff178640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f8161a20 con 0x7fc1f80b4160 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1f7fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f80aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1fd715640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc1f80aaa00 0x7fc1f8160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1f7fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f80aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:59596/0 (socket says 172.21.15.79:59596) 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1fcf14640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fc1f80b3d80 0x7fc1f80d2070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:45760/0 (socket says 172.21.15.79:45760) 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1f7fff640 1 -- 172.21.15.79:0/3863061060 learned_addr learned my addr 172.21.15.79:0/3863061060 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:18.046 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1f7fff640 1 -- 172.21.15.79:0/3863061060 >> v2:172.21.15.83:3300/0 conn(0x7fc1f80aaa00 msgr2=0x7fc1f8160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:18.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1f7fff640 1 --2- 172.21.15.79:0/3863061060 >> v2:172.21.15.83:3300/0 conn(0x7fc1f80aaa00 0x7fc1f8160ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1f7fff640 1 -- 172.21.15.79:0/3863061060 >> v1:172.21.15.83:6789/0 conn(0x7fc1f80b3d80 legacy=0x7fc1f80d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1f7fff640 1 -- 172.21.15.79:0/3863061060 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f8161d00 con 0x7fc1f80b4160 2024-04-26T19:51:18.047 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.045+0000 7fc1fd715640 1 --2- 172.21.15.79:0/3863061060 >> v2:172.21.15.83:3300/0 conn(0x7fc1f80aaa00 0x7fc1f8160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:18.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.046+0000 7fc1f7fff640 1 --2- 172.21.15.79:0/3863061060 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f80aa4c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009870 tx=0x7fc1ec02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c3f2cfb5e490802 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.046+0000 7fc1f77fe640 1 -- 172.21.15.79:0/3863061060 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec037070 con 0x7fc1f80b4160 2024-04-26T19:51:18.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.046+0000 7fc1f77fe640 1 -- 172.21.15.79:0/3863061060 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1ec032440 con 0x7fc1f80b4160 2024-04-26T19:51:18.048 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.046+0000 7fc1f77fe640 1 -- 172.21.15.79:0/3863061060 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ec032810 con 0x7fc1f80b4160 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.047+0000 7fc1ff178640 1 -- 172.21.15.79:0/3863061060 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 msgr2=0x7fc1f80aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.047+0000 7fc1ff178640 1 --2- 172.21.15.79:0/3863061060 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f80aa4c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009870 tx=0x7fc1ec02a270 comp rx=0 tx=0).stop 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.047+0000 7fc1ff178640 1 -- 172.21.15.79:0/3863061060 shutdown_connections 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.047+0000 7fc1ff178640 1 --2- 172.21.15.79:0/3863061060 >> v2:172.21.15.83:3300/0 conn(0x7fc1f80aaa00 0x7fc1f8160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.047+0000 7fc1ff178640 1 --2- 172.21.15.79:0/3863061060 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f80aa4c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.047+0000 7fc1ff178640 1 -- 172.21.15.79:0/3863061060 >> 172.21.15.79:0/3863061060 conn(0x7fc1f80b2720 msgr2=0x7fc1f8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.047+0000 7fc1ff178640 1 -- 172.21.15.79:0/3863061060 shutdown_connections 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.047+0000 7fc1ff178640 1 -- 172.21.15.79:0/3863061060 wait complete. 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.048+0000 7fc1ff178640 1 Processor -- start 2024-04-26T19:51:18.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.048+0000 7fc1ff178640 1 -- start start 2024-04-26T19:51:18.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1ff178640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1f80aaa00 0x7fc1f815a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1ff178640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80b3d80 0x7fc1f81537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1ff178640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f813fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1ff178640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f80d2780 con 0x7fc1f80aaa00 2024-04-26T19:51:18.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1ff178640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f80d1b50 con 0x7fc1f80b3d80 2024-04-26T19:51:18.051 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1ff178640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f80d2900 con 0x7fc1f80b4160 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fcf14640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1f80aaa00 0x7fc1f815a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fcf14640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1f80aaa00 0x7fc1f815a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56952/0 (socket says 172.21.15.79:56952) 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fcf14640 1 -- 172.21.15.79:0/786946013 learned_addr learned my addr 172.21.15.79:0/786946013 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1f7fff640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80b3d80 0x7fc1f81537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fd715640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f813fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fcf14640 1 -- 172.21.15.79:0/786946013 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 msgr2=0x7fc1f813fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fcf14640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f813fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fcf14640 1 -- 172.21.15.79:0/786946013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80b3d80 msgr2=0x7fc1f81537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fcf14640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80b3d80 0x7fc1f81537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fcf14640 1 -- 172.21.15.79:0/786946013 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f8140540 con 0x7fc1f80aaa00 2024-04-26T19:51:18.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1f7fff640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80b3d80 0x7fc1f81537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1fcf14640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1f80aaa00 0x7fc1f815a130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000c730 tx=0x7fc1f000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1f57fa640 1 -- 172.21.15.79:0/786946013 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0017070 con 0x7fc1f80aaa00 2024-04-26T19:51:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1f57fa640 1 -- 172.21.15.79:0/786946013 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1f000ce40 con 0x7fc1f80aaa00 2024-04-26T19:51:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f8140820 con 0x7fc1f80aaa00 2024-04-26T19:51:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1f57fa640 1 -- 172.21.15.79:0/786946013 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f000a860 con 0x7fc1f80aaa00 2024-04-26T19:51:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.049+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f8141460 con 0x7fc1f80aaa00 2024-04-26T19:51:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.050+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f80506c0 con 0x7fc1f80aaa00 2024-04-26T19:51:18.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.051+0000 7fc1f57fa640 1 -- 172.21.15.79:0/786946013 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1f001a070 con 0x7fc1f80aaa00 2024-04-26T19:51:18.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.051+0000 7fc1f57fa640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc1e406c8f0 0x7fc1e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.051+0000 7fc1f7fff640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc1e406c8f0 0x7fc1e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.052+0000 7fc1f57fa640 1 -- 172.21.15.79:0/786946013 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc1f008b4c0 con 0x7fc1f80aaa00 2024-04-26T19:51:18.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.052+0000 7fc1f7fff640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc1e406c8f0 0x7fc1e406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec034000 tx=0x7fc1ec02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.056+0000 7fc1f57fa640 1 -- 172.21.15.79:0/786946013 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc1f0059580 con 0x7fc1f80aaa00 2024-04-26T19:51:18.232 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.231+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc1f81416a0 con 0x7fc1f80aaa00 2024-04-26T19:51:18.233 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.232+0000 7fc1f57fa640 1 -- 172.21.15.79:0/786946013 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7fc1f81416a0 con 0x7fc1f80aaa00 2024-04-26T19:51:18.233 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:18.233 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:18.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc1e406c8f0 msgr2=0x7fc1e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc1e406c8f0 0x7fc1e406edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec034000 tx=0x7fc1ec02e000 comp rx=0 tx=0).stop 2024-04-26T19:51:18.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1f80aaa00 msgr2=0x7fc1f815a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1f80aaa00 0x7fc1f815a130 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc1f000c730 tx=0x7fc1f000cbf0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 shutdown_connections 2024-04-26T19:51:18.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc1e406c8f0 0x7fc1e406edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc1f80b4160 0x7fc1f813fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc1f80b3d80 0x7fc1f81537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 --2- 172.21.15.79:0/786946013 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc1f80aaa00 0x7fc1f815a130 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 >> 172.21.15.79:0/786946013 conn(0x7fc1f80b2720 msgr2=0x7fc1f806fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:18.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 shutdown_connections 2024-04-26T19:51:18.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.234+0000 7fc1ff178640 1 -- 172.21.15.79:0/786946013 wait complete. 2024-04-26T19:51:18.247 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:18.248 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:18.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.348+0000 7f46efdbf640 1 Processor -- start 2024-04-26T19:51:18.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.348+0000 7f46efdbf640 1 -- start start 2024-04-26T19:51:18.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46efdbf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 0x7f46e806cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46efdbf640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f46e806f0a0 0x7f46e806d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46efdbf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46e8160ff0 con 0x7f46e815d220 2024-04-26T19:51:18.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46efdbf640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f46e81612d0 con 0x7f46e806f0a0 2024-04-26T19:51:18.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46efdbf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f46e81615b0 con 0x7f46e80cf760 2024-04-26T19:51:18.355 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46ee35c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f46e806f0a0 0x7f46e806d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46ee35c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f46e806f0a0 0x7f46e806d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56964/0 (socket says 172.21.15.79:56964) 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46ee35c640 1 -- 172.21.15.79:0/322068327 learned_addr learned my addr 172.21.15.79:0/322068327 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46ed35a640 1 --2- 172.21.15.79:0/322068327 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 0x7f46e806cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46ee35c640 1 -- 172.21.15.79:0/322068327 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 msgr2=0x7f46e806cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46ee35c640 1 --2- 172.21.15.79:0/322068327 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 0x7f46e806cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46ee35c640 1 -- 172.21.15.79:0/322068327 >> v1:172.21.15.83:6789/0 conn(0x7f46e815d220 legacy=0x7f46e80cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46ee35c640 1 -- 172.21.15.79:0/322068327 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e8161890 con 0x7f46e806f0a0 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.349+0000 7f46ed35a640 1 --2- 172.21.15.79:0/322068327 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 0x7f46e806cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46ee35c640 1 --2- 172.21.15.79:0/322068327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 0x7f46e806d430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009870 tx=0x7f46d802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80616356f1fe49bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46ecb59640 1 -- 172.21.15.79:0/322068327 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8037070 con 0x7f46e806f0a0 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46ecb59640 1 -- 172.21.15.79:0/322068327 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46d8032440 con 0x7f46e806f0a0 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46ecb59640 1 -- 172.21.15.79:0/322068327 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46d8032810 con 0x7f46e806f0a0 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46efdbf640 1 -- 172.21.15.79:0/322068327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 msgr2=0x7f46e806d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.356 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46efdbf640 1 --2- 172.21.15.79:0/322068327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 0x7f46e806d430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009870 tx=0x7f46d802a180 comp rx=0 tx=0).stop 2024-04-26T19:51:18.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46efdbf640 1 -- 172.21.15.79:0/322068327 shutdown_connections 2024-04-26T19:51:18.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46efdbf640 1 --2- 172.21.15.79:0/322068327 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 0x7f46e806d430 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46efdbf640 1 --2- 172.21.15.79:0/322068327 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 0x7f46e806cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.350+0000 7f46efdbf640 1 -- 172.21.15.79:0/322068327 >> 172.21.15.79:0/322068327 conn(0x7f46e80d24f0 msgr2=0x7f46e80d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:18.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.351+0000 7f46efdbf640 1 -- 172.21.15.79:0/322068327 shutdown_connections 2024-04-26T19:51:18.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.351+0000 7f46efdbf640 1 -- 172.21.15.79:0/322068327 wait complete. 2024-04-26T19:51:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46efdbf640 1 Processor -- start 2024-04-26T19:51:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46efdbf640 1 -- start start 2024-04-26T19:51:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46efdbf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 0x7f46e8150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46efdbf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 0x7f46e815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46efdbf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e815d220 0x7f46e815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46efdbf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f46e815db70 con 0x7f46e806f0a0 2024-04-26T19:51:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46efdbf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f46e815d9f0 con 0x7f46e815d220 2024-04-26T19:51:18.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46efdbf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f46e815dcf0 con 0x7f46e80cf760 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46edb5b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 0x7f46e8150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46edb5b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 0x7f46e8150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56974/0 (socket says 172.21.15.79:56974) 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.352+0000 7f46edb5b640 1 -- 172.21.15.79:0/805323177 learned_addr learned my addr 172.21.15.79:0/805323177 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46ed35a640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 0x7f46e815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46ee35c640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e815d220 0x7f46e815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46edb5b640 1 -- 172.21.15.79:0/805323177 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 msgr2=0x7f46e815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46edb5b640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 0x7f46e815b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46edb5b640 1 -- 172.21.15.79:0/805323177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e815d220 msgr2=0x7f46e815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46edb5b640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e815d220 0x7f46e815b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46edb5b640 1 -- 172.21.15.79:0/805323177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46e8155600 con 0x7f46e806f0a0 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46edb5b640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 0x7f46e8150480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f46e000ba80 tx=0x7f46e000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46e67fc640 1 -- 172.21.15.79:0/805323177 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e000cc00 con 0x7f46e806f0a0 2024-04-26T19:51:18.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46e67fc640 1 -- 172.21.15.79:0/805323177 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46e000cda0 con 0x7f46e806f0a0 2024-04-26T19:51:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46e81558e0 con 0x7f46e806f0a0 2024-04-26T19:51:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46e67fc640 1 -- 172.21.15.79:0/805323177 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0003bb0 con 0x7f46e806f0a0 2024-04-26T19:51:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.353+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46e8155ea0 con 0x7f46e806f0a0 2024-04-26T19:51:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.354+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46e81f3040 con 0x7f46e806f0a0 2024-04-26T19:51:18.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.354+0000 7f46e67fc640 1 -- 172.21.15.79:0/805323177 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46e0014440 con 0x7f46e806f0a0 2024-04-26T19:51:18.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.355+0000 7f46e67fc640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f46d406c8f0 0x7f46d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.355+0000 7f46ed35a640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f46d406c8f0 0x7f46d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.355+0000 7f46e67fc640 1 -- 172.21.15.79:0/805323177 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f46e008bf30 con 0x7f46e806f0a0 2024-04-26T19:51:18.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.355+0000 7f46ed35a640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f46d406c8f0 0x7f46d406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f46e8144810 tx=0x7f46dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.359+0000 7f46e67fc640 1 -- 172.21.15.79:0/805323177 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f46e0059ed0 con 0x7f46e806f0a0 2024-04-26T19:51:18.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.534+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46e806c9a0 con 0x7f46e806f0a0 2024-04-26T19:51:18.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.535+0000 7f46e67fc640 1 -- 172.21.15.79:0/805323177 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f46e00597d0 con 0x7f46e806f0a0 2024-04-26T19:51:18.535 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f46d406c8f0 msgr2=0x7f46d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f46d406c8f0 0x7f46d406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f46e8144810 tx=0x7f46dc008040 comp rx=0 tx=0).stop 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 msgr2=0x7f46e8150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 0x7f46e8150480 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f46e000ba80 tx=0x7f46e000bf40 comp rx=0 tx=0).stop 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 shutdown_connections 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f46d406c8f0 0x7f46d406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f46e815d220 0x7f46e815b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f46e80cf760 0x7f46e815b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 --2- 172.21.15.79:0/805323177 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f46e806f0a0 0x7f46e8150480 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 >> 172.21.15.79:0/805323177 conn(0x7f46e80d24f0 msgr2=0x7f46e8158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 shutdown_connections 2024-04-26T19:51:18.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.537+0000 7f46efdbf640 1 -- 172.21.15.79:0/805323177 wait complete. 2024-04-26T19:51:18.548 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:18.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.647+0000 7fd8b6a2a640 1 Processor -- start 2024-04-26T19:51:18.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.647+0000 7fd8b6a2a640 1 -- start start 2024-04-26T19:51:18.650 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8b6a2a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd8b0148f30 0x7fd8b00b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8b6a2a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8b00b7620 0x7fd8b00b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8b6a2a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b00b7e60 con 0x7fd8b006c350 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8b6a2a640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b00b8140 con 0x7fd8b00b7620 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8b6a2a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd8b00b8420 con 0x7fd8b0148f30 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8af7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8b00b7620 0x7fd8b00b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8af7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd8b00b7620 0x7fd8b00b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:56986/0 (socket says 172.21.15.79:56986) 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8af7fe640 1 -- 172.21.15.79:0/3971410700 learned_addr learned my addr 172.21.15.79:0/3971410700 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8affff640 1 --2- 172.21.15.79:0/3971410700 >> v2:172.21.15.83:3300/0 conn(0x7fd8b0148f30 0x7fd8b00b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8af7fe640 1 -- 172.21.15.79:0/3971410700 >> v2:172.21.15.83:3300/0 conn(0x7fd8b0148f30 msgr2=0x7fd8b00b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8af7fe640 1 --2- 172.21.15.79:0/3971410700 >> v2:172.21.15.83:3300/0 conn(0x7fd8b0148f30 0x7fd8b00b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.648+0000 7fd8af7fe640 1 -- 172.21.15.79:0/3971410700 >> v1:172.21.15.83:6789/0 conn(0x7fd8b006c350 legacy=0x7fd8b015fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.649+0000 7fd8af7fe640 1 -- 172.21.15.79:0/3971410700 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b00b8700 con 0x7fd8b00b7620 2024-04-26T19:51:18.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.649+0000 7fd8af7fe640 1 --2- 172.21.15.79:0/3971410700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b00b7620 0x7fd8b00b8f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00748b0 tx=0x7fd8a402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63afedeb579bb56d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.649+0000 7fd8aeffd640 1 -- 172.21.15.79:0/3971410700 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4037070 con 0x7fd8b00b7620 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.649+0000 7fd8aeffd640 1 -- 172.21.15.79:0/3971410700 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a4032440 con 0x7fd8b00b7620 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.649+0000 7fd8aeffd640 1 -- 172.21.15.79:0/3971410700 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4032810 con 0x7fd8b00b7620 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.650+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/3971410700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b00b7620 msgr2=0x7fd8b00b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.650+0000 7fd8b6a2a640 1 --2- 172.21.15.79:0/3971410700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b00b7620 0x7fd8b00b8f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00748b0 tx=0x7fd8a402a230 comp rx=0 tx=0).stop 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.650+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/3971410700 shutdown_connections 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.650+0000 7fd8b6a2a640 1 --2- 172.21.15.79:0/3971410700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b00b7620 0x7fd8b00b8f00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.650+0000 7fd8b6a2a640 1 --2- 172.21.15.79:0/3971410700 >> v2:172.21.15.83:3300/0 conn(0x7fd8b0148f30 0x7fd8b00b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.650+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/3971410700 >> 172.21.15.79:0/3971410700 conn(0x7fd8b0073b40 msgr2=0x7fd8b0073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.650+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/3971410700 shutdown_connections 2024-04-26T19:51:18.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.650+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/3971410700 wait complete. 2024-04-26T19:51:18.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.651+0000 7fd8b6a2a640 1 Processor -- start 2024-04-26T19:51:18.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.651+0000 7fd8b6a2a640 1 -- start start 2024-04-26T19:51:18.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b6a2a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8b006c350 0x7fd8b013d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b6a2a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b00b7620 0x7fd8b012d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b6a2a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b0148f30 0x7fd8b0143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b6a2a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b00b7a60 con 0x7fd8b0148f30 2024-04-26T19:51:18.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b6a2a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b00b9440 con 0x7fd8b00b7620 2024-04-26T19:51:18.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b6a2a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b00b7be0 con 0x7fd8b006c350 2024-04-26T19:51:18.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b4fc7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b0148f30 0x7fd8b0143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b4fc7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b0148f30 0x7fd8b0143240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57000/0 (socket says 172.21.15.79:57000) 2024-04-26T19:51:18.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b4fc7640 1 -- 172.21.15.79:0/1116436287 learned_addr learned my addr 172.21.15.79:0/1116436287 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:18.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b4fc7640 1 -- 172.21.15.79:0/1116436287 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8b006c350 msgr2=0x7fd8b013d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b4fc7640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8b006c350 0x7fd8b013d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8af7fe640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b00b7620 0x7fd8b012d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b4fc7640 1 -- 172.21.15.79:0/1116436287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b00b7620 msgr2=0x7fd8b012d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b4fc7640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b00b7620 0x7fd8b012d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8b4fc7640 1 -- 172.21.15.79:0/1116436287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b0140120 con 0x7fd8b0148f30 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.652+0000 7fd8af7fe640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b00b7620 0x7fd8b012d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.653+0000 7fd8b4fc7640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b0148f30 0x7fd8b0143240 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c7b0 tx=0x7fd8a800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.653+0000 7fd8acff9640 1 -- 172.21.15.79:0/1116436287 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8004b60 con 0x7fd8b0148f30 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.653+0000 7fd8acff9640 1 -- 172.21.15.79:0/1116436287 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a8004d00 con 0x7fd8b0148f30 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.653+0000 7fd8acff9640 1 -- 172.21.15.79:0/1116436287 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800aa90 con 0x7fd8b0148f30 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.653+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b015c9b0 con 0x7fd8b0148f30 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.653+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b0143780 con 0x7fd8b0148f30 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.654+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b015c7a0 con 0x7fd8b0148f30 2024-04-26T19:51:18.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.654+0000 7fd8acff9640 1 -- 172.21.15.79:0/1116436287 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd8a800ac30 con 0x7fd8b0148f30 2024-04-26T19:51:18.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.655+0000 7fd8acff9640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd89c06c8f0 0x7fd89c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.655+0000 7fd8acff9640 1 -- 172.21.15.79:0/1116436287 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd8a808c1c0 con 0x7fd8b0148f30 2024-04-26T19:51:18.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.655+0000 7fd8affff640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd89c06c8f0 0x7fd89c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.655+0000 7fd8affff640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd89c06c8f0 0x7fd89c06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000a810 tx=0x7fd8a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.659+0000 7fd8acff9640 1 -- 172.21.15.79:0/1116436287 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd8a8020070 con 0x7fd8b0148f30 2024-04-26T19:51:18.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.834+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd8b00506c0 con 0x7fd8b0148f30 2024-04-26T19:51:18.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.834+0000 7fd8acff9640 1 -- 172.21.15.79:0/1116436287 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fd8a805a280 con 0x7fd8b0148f30 2024-04-26T19:51:18.835 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd89c06c8f0 msgr2=0x7fd89c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd89c06c8f0 0x7fd89c06edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000a810 tx=0x7fd8a0008040 comp rx=0 tx=0).stop 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b0148f30 msgr2=0x7fd8b0143240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b0148f30 0x7fd8b0143240 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd8a800c7b0 tx=0x7fd8a800cc70 comp rx=0 tx=0).stop 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 shutdown_connections 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd89c06c8f0 0x7fd89c06edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd8b0148f30 0x7fd8b0143240 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd8b00b7620 0x7fd8b012d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 --2- 172.21.15.79:0/1116436287 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd8b006c350 0x7fd8b013d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 >> 172.21.15.79:0/1116436287 conn(0x7fd8b0073b40 msgr2=0x7fd8b00b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:18.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 shutdown_connections 2024-04-26T19:51:18.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.836+0000 7fd8b6a2a640 1 -- 172.21.15.79:0/1116436287 wait complete. 2024-04-26T19:51:18.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:18.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.948+0000 7f23ad322640 1 Processor -- start 2024-04-26T19:51:18.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.948+0000 7f23ad322640 1 -- start start 2024-04-26T19:51:18.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.949+0000 7f23ad322640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a8141fb0 0x7f23a8160420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.949+0000 7f23ad322640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f23a8163530 0x7f23a80b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.949+0000 7f23ad322640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23a806de20 con 0x7f23a8142390 2024-04-26T19:51:18.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.949+0000 7f23ad322640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f23a81639e0 con 0x7f23a8163530 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.949+0000 7f23ad322640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f23a8163cc0 con 0x7f23a8141fb0 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.949+0000 7f23a67fc640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f23a8142390 0x7f23a8162e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33226/0 (socket says 172.21.15.79:33226) 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.949+0000 7f23a67fc640 1 -- 172.21.15.79:0/2039370930 learned_addr learned my addr 172.21.15.79:0/2039370930 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.949+0000 7f23a77fe640 1 --2- 172.21.15.79:0/2039370930 >> v2:172.21.15.83:3300/0 conn(0x7f23a8163530 0x7f23a80b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.949+0000 7f23a6ffd640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a8141fb0 0x7f23a8160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357664914 0 0) 0x7f23a806de20 con 0x7f23a8142390 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2390003520 con 0x7f23a8142390 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2596203044 0 0) 0x7f2390003520 con 0x7f23a8142390 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a8141fb0 msgr2=0x7f23a8160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a8141fb0 0x7f23a8160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 >> v2:172.21.15.83:3300/0 conn(0x7f23a8163530 msgr2=0x7f23a80b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 --2- 172.21.15.79:0/2039370930 >> v2:172.21.15.83:3300/0 conn(0x7f23a8163530 0x7f23a80b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a815f940 con 0x7f23a8142390 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f239c001670 con 0x7f23a8142390 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a6ffd640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a8141fb0 0x7f23a8160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 >> v1:172.21.15.79:6789/0 conn(0x7f23a8142390 legacy=0x7f23a8162e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a8141fb0 0x7f23900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2390003ef0 0x7f2390004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390004d40 0x7f23900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81639e0 con 0x7f2390004d40 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23a8163cc0 con 0x7f2390003ef0 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f239000b9c0 con 0x7f23a8141fb0 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a77fe640 1 --2- 172.21.15.79:0/2039370930 >> v2:172.21.15.83:3300/0 conn(0x7f23a8163530 0x7f23a80b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a77fe640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390004d40 0x7f23900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a6ffd640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a8141fb0 0x7f23900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a67fc640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2390003ef0 0x7f2390004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a77fe640 1 -- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a8141fb0 msgr2=0x7f23900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a77fe640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a8141fb0 0x7f23900039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a77fe640 1 -- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2390003ef0 msgr2=0x7f2390004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a77fe640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2390003ef0 0x7f2390004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.950+0000 7f23a77fe640 1 -- 172.21.15.79:0/2039370930 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2390003520 con 0x7f2390004d40 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.951+0000 7f23a77fe640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390004d40 0x7f23900051c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009c40 tx=0x7f2390005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcfb1aca9f4f7550 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.951+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a0009840 con 0x7f2390004d40 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.951+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23a0004020 con 0x7f2390004d40 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.951+0000 7f23a5ffb640 1 -- 172.21.15.79:0/2039370930 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23a00043d0 con 0x7f2390004d40 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.952+0000 7f23ad322640 1 -- 172.21.15.79:0/2039370930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390004d40 msgr2=0x7f23900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.952+0000 7f23ad322640 1 --2- 172.21.15.79:0/2039370930 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2390004d40 0x7f23900051c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f23a0009c40 tx=0x7f2390005830 comp rx=0 tx=0).stop 2024-04-26T19:51:18.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.952+0000 7f23a77fe640 1 -- 172.21.15.79:0/2039370930 reap_dead start 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.952+0000 7f23ad322640 1 -- 172.21.15.79:0/2039370930 shutdown_connections 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.952+0000 7f23ad322640 1 -- 172.21.15.79:0/2039370930 >> 172.21.15.79:0/2039370930 conn(0x7f23a8068390 msgr2=0x7f23a80687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.952+0000 7f23ad322640 1 -- 172.21.15.79:0/2039370930 shutdown_connections 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.952+0000 7f23ad322640 1 -- 172.21.15.79:0/2039370930 wait complete. 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.953+0000 7f23ad322640 1 Processor -- start 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.953+0000 7f23ad322640 1 -- start start 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23ad322640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a81eaea0 0x7f23a8158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23ad322640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23a80afb10 0x7f23a81591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23ad322640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a81e8da0 0x7f23a81e9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23ad322640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f23a81605a0 con 0x7f23a81eaea0 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23ad322640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f23a8160420 con 0x7f23a80afb10 2024-04-26T19:51:18.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23ad322640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f23a8160720 con 0x7f23a81e8da0 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a81eaea0 0x7f23a8158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a81eaea0 0x7f23a8158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57008/0 (socket says 172.21.15.79:57008) 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a6ffd640 1 -- 172.21.15.79:0/951931788 learned_addr learned my addr 172.21.15.79:0/951931788 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a77fe640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a81e8da0 0x7f23a81e9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a67fc640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23a80afb10 0x7f23a81591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a6ffd640 1 -- 172.21.15.79:0/951931788 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a81e8da0 msgr2=0x7f23a81e9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a6ffd640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a81e8da0 0x7f23a81e9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a6ffd640 1 -- 172.21.15.79:0/951931788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23a80afb10 msgr2=0x7f23a81591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a6ffd640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23a80afb10 0x7f23a81591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a6ffd640 1 -- 172.21.15.79:0/951931788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a81e9b40 con 0x7f23a81eaea0 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a77fe640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a81e8da0 0x7f23a81e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23a6ffd640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a81eaea0 0x7f23a8158c60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2398011e30 tx=0x7f239800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f238f7fe640 1 -- 172.21.15.79:0/951931788 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398003b90 con 0x7f23a81eaea0 2024-04-26T19:51:18.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f238f7fe640 1 -- 172.21.15.79:0/951931788 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2398003d30 con 0x7f23a81eaea0 2024-04-26T19:51:18.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a81e9e20 con 0x7f23a81eaea0 2024-04-26T19:51:18.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f238f7fe640 1 -- 172.21.15.79:0/951931788 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2398004270 con 0x7f23a81eaea0 2024-04-26T19:51:18.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.954+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a81e9760 con 0x7f23a81eaea0 2024-04-26T19:51:18.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.955+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a80506c0 con 0x7f23a81eaea0 2024-04-26T19:51:18.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.956+0000 7f238f7fe640 1 -- 172.21.15.79:0/951931788 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2398004410 con 0x7f23a81eaea0 2024-04-26T19:51:18.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.956+0000 7f238f7fe640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f239406c8f0 0x7f239406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:18.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.956+0000 7f238f7fe640 1 -- 172.21.15.79:0/951931788 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f239808cec0 con 0x7f23a81eaea0 2024-04-26T19:51:18.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.956+0000 7f23a67fc640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f239406c8f0 0x7f239406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:18.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.957+0000 7f23a67fc640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f239406c8f0 0x7f239406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f239c004840 tx=0x7f239c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:18.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:18.960+0000 7f238f7fe640 1 -- 172.21.15.79:0/951931788 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2398019070 con 0x7f23a81eaea0 2024-04-26T19:51:19.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.137+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f23a815f0f0 con 0x7f23a81eaea0 2024-04-26T19:51:19.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.137+0000 7f238f7fe640 1 -- 172.21.15.79:0/951931788 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f2398010040 con 0x7f23a81eaea0 2024-04-26T19:51:19.138 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:19.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f239406c8f0 msgr2=0x7f239406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f239406c8f0 0x7f239406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f239c004840 tx=0x7f239c009040 comp rx=0 tx=0).stop 2024-04-26T19:51:19.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a81eaea0 msgr2=0x7f23a8158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a81eaea0 0x7f23a8158c60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2398011e30 tx=0x7f239800f580 comp rx=0 tx=0).stop 2024-04-26T19:51:19.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 shutdown_connections 2024-04-26T19:51:19.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f239406c8f0 0x7f239406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f23a81e8da0 0x7f23a81e9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f23a80afb10 0x7f23a81591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 --2- 172.21.15.79:0/951931788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f23a81eaea0 0x7f23a8158c60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 >> 172.21.15.79:0/951931788 conn(0x7f23a8068390 msgr2=0x7f23a806f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:19.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.139+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 shutdown_connections 2024-04-26T19:51:19.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.140+0000 7f23ad322640 1 -- 172.21.15.79:0/951931788 wait complete. 2024-04-26T19:51:19.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:19.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.251+0000 7f6cfa146640 1 Processor -- start 2024-04-26T19:51:19.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.251+0000 7f6cfa146640 1 -- start start 2024-04-26T19:51:19.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.251+0000 7f6cfa146640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf415c1e0 0x7f6cf4073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.251+0000 7f6cfa146640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6cf4072000 0x7f6cf406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.251+0000 7f6cfa146640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf4072620 con 0x7f6cf406f920 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cfa146640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf40702c0 con 0x7f6cf4072000 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cfa146640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf40730d0 con 0x7f6cf415c1e0 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf2ffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6cf406f920 0x7f6cf40718f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33236/0 (socket says 172.21.15.79:33236) 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf2ffd640 1 -- 172.21.15.79:0/2152832871 learned_addr learned my addr 172.21.15.79:0/2152832871 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf37fe640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf415c1e0 0x7f6cf4073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf3fff640 1 --2- 172.21.15.79:0/2152832871 >> v2:172.21.15.83:3300/0 conn(0x7f6cf4072000 0x7f6cf406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2264233606 0 0) 0x7f6cf4072620 con 0x7f6cf406f920 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6cdc003520 con 0x7f6cf406f920 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3109906276 0 0) 0x7f6cdc003520 con 0x7f6cf406f920 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf415c1e0 msgr2=0x7f6cf4073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf415c1e0 0x7f6cf4073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 >> v2:172.21.15.83:3300/0 conn(0x7f6cf4072000 msgr2=0x7f6cf406fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:19.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 --2- 172.21.15.79:0/2152832871 >> v2:172.21.15.83:3300/0 conn(0x7f6cf4072000 0x7f6cf406fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf40733b0 con 0x7f6cf406f920 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f6ce8001670 con 0x7f6cf406f920 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf3fff640 1 --2- 172.21.15.79:0/2152832871 >> v2:172.21.15.83:3300/0 conn(0x7f6cf4072000 0x7f6cf406fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 >> v1:172.21.15.79:6789/0 conn(0x7f6cf406f920 legacy=0x7f6cf40718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cdc0039b0 0x7f6cdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc004800 0x7f6cdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf415c1e0 0x7f6cdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf40702c0 con 0x7f6cdc0039b0 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf40730d0 con 0x7f6cdc004800 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f6cdc00ba40 con 0x7f6cf415c1e0 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf37fe640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cdc0039b0 0x7f6cdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf2ffd640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc004800 0x7f6cdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.252+0000 7f6cf3fff640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf415c1e0 0x7f6cdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.253+0000 7f6cf37fe640 1 -- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf415c1e0 msgr2=0x7f6cdc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.253+0000 7f6cf37fe640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf415c1e0 0x7f6cdc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.253+0000 7f6cf37fe640 1 -- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc004800 msgr2=0x7f6cdc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.253+0000 7f6cf37fe640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc004800 0x7f6cdc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.253+0000 7f6cf37fe640 1 -- 172.21.15.79:0/2152832871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc003520 con 0x7f6cdc0039b0 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.253+0000 7f6cf37fe640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cdc0039b0 0x7f6cdc003dd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009ac0 tx=0x7f6cdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223b6d024df62efd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.253+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4036070 con 0x7f6cdc0039b0 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.253+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce4031400 con 0x7f6cdc0039b0 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.253+0000 7f6cf27fc640 1 -- 172.21.15.79:0/2152832871 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4009690 con 0x7f6cdc0039b0 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.254+0000 7f6cfa146640 1 -- 172.21.15.79:0/2152832871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cdc0039b0 msgr2=0x7f6cdc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.254+0000 7f6cfa146640 1 --2- 172.21.15.79:0/2152832871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cdc0039b0 0x7f6cdc003dd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009ac0 tx=0x7f6cdc005830 comp rx=0 tx=0).stop 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.254+0000 7f6cf3fff640 1 -- 172.21.15.79:0/2152832871 reap_dead start 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.254+0000 7f6cfa146640 1 -- 172.21.15.79:0/2152832871 shutdown_connections 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.254+0000 7f6cfa146640 1 -- 172.21.15.79:0/2152832871 >> 172.21.15.79:0/2152832871 conn(0x7f6cf40b2540 msgr2=0x7f6cf40b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.254+0000 7f6cfa146640 1 -- 172.21.15.79:0/2152832871 shutdown_connections 2024-04-26T19:51:19.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.254+0000 7f6cfa146640 1 -- 172.21.15.79:0/2152832871 wait complete. 2024-04-26T19:51:19.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.255+0000 7f6cfa146640 1 Processor -- start 2024-04-26T19:51:19.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.255+0000 7f6cfa146640 1 -- start start 2024-04-26T19:51:19.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cfa146640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc0039b0 0x7f6cf414bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cfa146640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf4146f70 0x7f6cf40b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cfa146640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cf40b41e0 0x7f6cf41358f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cfa146640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf4073d30 con 0x7f6cf40b41e0 2024-04-26T19:51:19.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cfa146640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf4073bb0 con 0x7f6cdc0039b0 2024-04-26T19:51:19.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cfa146640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf4073eb0 con 0x7f6cf4146f70 2024-04-26T19:51:19.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cf40b41e0 0x7f6cf41358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cf40b41e0 0x7f6cf41358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57028/0 (socket says 172.21.15.79:57028) 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf3fff640 1 -- 172.21.15.79:0/3843853085 learned_addr learned my addr 172.21.15.79:0/3843853085 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf37fe640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc0039b0 0x7f6cf414bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf2ffd640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf4146f70 0x7f6cf40b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf3fff640 1 -- 172.21.15.79:0/3843853085 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf4146f70 msgr2=0x7f6cf40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf3fff640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf4146f70 0x7f6cf40b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf3fff640 1 -- 172.21.15.79:0/3843853085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc0039b0 msgr2=0x7f6cf414bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf3fff640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc0039b0 0x7f6cf414bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf3fff640 1 -- 172.21.15.79:0/3843853085 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf414ae50 con 0x7f6cf40b41e0 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.256+0000 7f6cf3fff640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cf40b41e0 0x7f6cf41358f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40b0460 tx=0x7f6cec00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.257+0000 7f6cdbfff640 1 -- 172.21.15.79:0/3843853085 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec003b90 con 0x7f6cf40b41e0 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.257+0000 7f6cdbfff640 1 -- 172.21.15.79:0/3843853085 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cec003d30 con 0x7f6cf40b41e0 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.257+0000 7f6cdbfff640 1 -- 172.21.15.79:0/3843853085 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec004260 con 0x7f6cf40b41e0 2024-04-26T19:51:19.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.257+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf414b130 con 0x7f6cf40b41e0 2024-04-26T19:51:19.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.257+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf414b6f0 con 0x7f6cf40b41e0 2024-04-26T19:51:19.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.258+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf40b4660 con 0x7f6cf40b41e0 2024-04-26T19:51:19.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.260+0000 7f6cdbfff640 1 -- 172.21.15.79:0/3843853085 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cec004400 con 0x7f6cf40b41e0 2024-04-26T19:51:19.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.260+0000 7f6cdbfff640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6ce006c8f0 0x7f6ce006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.260+0000 7f6cdbfff640 1 -- 172.21.15.79:0/3843853085 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6cec08cee0 con 0x7f6cf40b41e0 2024-04-26T19:51:19.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.260+0000 7f6cf37fe640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6ce006c8f0 0x7f6ce006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.261+0000 7f6cf37fe640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6ce006c8f0 0x7f6ce006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4033000 tx=0x7f6ce4008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:19.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.264+0000 7f6cdbfff640 1 -- 172.21.15.79:0/3843853085 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6cec019030 con 0x7f6cf40b41e0 2024-04-26T19:51:19.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.437+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6cf40506c0 con 0x7f6cf40b41e0 2024-04-26T19:51:19.438 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.437+0000 7f6cdbfff640 1 -- 172.21.15.79:0/3843853085 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f6cec010040 con 0x7f6cf40b41e0 2024-04-26T19:51:19.438 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:19.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6ce006c8f0 msgr2=0x7f6ce006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6ce006c8f0 0x7f6ce006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4033000 tx=0x7f6ce4008030 comp rx=0 tx=0).stop 2024-04-26T19:51:19.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cf40b41e0 msgr2=0x7f6cf41358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cf40b41e0 0x7f6cf41358f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40b0460 tx=0x7f6cec00f580 comp rx=0 tx=0).stop 2024-04-26T19:51:19.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 shutdown_connections 2024-04-26T19:51:19.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f6ce006c8f0 0x7f6ce006edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6cf40b41e0 0x7f6cf41358f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6cf4146f70 0x7f6cf40b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 --2- 172.21.15.79:0/3843853085 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6cdc0039b0 0x7f6cf414bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.439+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 >> 172.21.15.79:0/3843853085 conn(0x7f6cf40b2540 msgr2=0x7f6cf415af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:19.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.440+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 shutdown_connections 2024-04-26T19:51:19.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.440+0000 7f6cfa146640 1 -- 172.21.15.79:0/3843853085 wait complete. 2024-04-26T19:51:19.451 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-04-26T19:51:19.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.548+0000 7f75bedc1640 1 Processor -- start 2024-04-26T19:51:19.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.548+0000 7f75bedc1640 1 -- start start 2024-04-26T19:51:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75bedc1640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f75b80b44f0 0x7f75b8166e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75bedc1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b80ce1a0 con 0x7f75b80cd9c0 2024-04-26T19:51:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75bedc1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75b80cdec0 con 0x7f75b8156670 2024-04-26T19:51:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75bedc1640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f75b8073110 con 0x7f75b80b44f0 2024-04-26T19:51:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75bd35e640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f75b80b44f0 0x7f75b8166e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75b7fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f75b80cd9c0 0x7f75b80b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33246/0 (socket says 172.21.15.79:33246) 2024-04-26T19:51:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75b7fff640 1 -- 172.21.15.79:0/335555574 learned_addr learned my addr 172.21.15.79:0/335555574 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75bd35e640 1 -- 172.21.15.79:0/335555574 >> v1:172.21.15.83:6789/0 conn(0x7f75b8156670 legacy=0x7f75b815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75bd35e640 1 -- 172.21.15.79:0/335555574 >> v1:172.21.15.79:6789/0 conn(0x7f75b80cd9c0 legacy=0x7f75b80b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75bd35e640 1 -- 172.21.15.79:0/335555574 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b8072ac0 con 0x7f75b80b44f0 2024-04-26T19:51:19.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.549+0000 7f75b77fe640 1 -- 172.21.15.79:0/335555574 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3489255106 0 0) 0x7f75b80ce1a0 con 0x7f75b80cd9c0 2024-04-26T19:51:19.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.550+0000 7f75bd35e640 1 --2- 172.21.15.79:0/335555574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80b44f0 0x7f75b8166e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009d90 tx=0x7f75b002c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba1680254e218b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:19.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.550+0000 7f75b77fe640 1 -- 172.21.15.79:0/335555574 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0036020 con 0x7f75b80b44f0 2024-04-26T19:51:19.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.550+0000 7f75b77fe640 1 -- 172.21.15.79:0/335555574 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b0032800 con 0x7f75b80b44f0 2024-04-26T19:51:19.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.550+0000 7f75b77fe640 1 -- 172.21.15.79:0/335555574 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b0032bd0 con 0x7f75b80b44f0 2024-04-26T19:51:19.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.551+0000 7f75bedc1640 1 -- 172.21.15.79:0/335555574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80b44f0 msgr2=0x7f75b8166e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.551+0000 7f75bedc1640 1 --2- 172.21.15.79:0/335555574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80b44f0 0x7f75b8166e00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009d90 tx=0x7f75b002c900 comp rx=0 tx=0).stop 2024-04-26T19:51:19.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.551+0000 7f75bedc1640 1 -- 172.21.15.79:0/335555574 shutdown_connections 2024-04-26T19:51:19.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.551+0000 7f75bedc1640 1 --2- 172.21.15.79:0/335555574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80b44f0 0x7f75b8166e00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.551+0000 7f75bedc1640 1 -- 172.21.15.79:0/335555574 >> 172.21.15.79:0/335555574 conn(0x7f75b80b39d0 msgr2=0x7f75b806feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.551+0000 7f75bedc1640 1 -- 172.21.15.79:0/335555574 shutdown_connections 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.551+0000 7f75bedc1640 1 -- 172.21.15.79:0/335555574 wait complete. 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.552+0000 7f75bedc1640 1 Processor -- start 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.552+0000 7f75bedc1640 1 -- start start 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75bedc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b80b44f0 0x7f75b8147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75bedc1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80cd9c0 0x7f75b806f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75bedc1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f75b8156670 0x7f75b8145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75bedc1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8156bd0 con 0x7f75b80cd9c0 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75bedc1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8156a50 con 0x7f75b80b44f0 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75bedc1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f75b80735a0 con 0x7f75b8156670 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80cd9c0 0x7f75b806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80cd9c0 0x7f75b806f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57046/0 (socket says 172.21.15.79:57046) 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b7fff640 1 -- 172.21.15.79:0/3830552767 learned_addr learned my addr 172.21.15.79:0/3830552767 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:19.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75bd35e640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f75b8156670 0x7f75b8145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75bcb5d640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b80b44f0 0x7f75b8147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b7fff640 1 -- 172.21.15.79:0/3830552767 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f75b8156670 msgr2=0x7f75b8145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b7fff640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f75b8156670 0x7f75b8145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b7fff640 1 -- 172.21.15.79:0/3830552767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b80b44f0 msgr2=0x7f75b8147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b7fff640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b80b44f0 0x7f75b8147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b7fff640 1 -- 172.21.15.79:0/3830552767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b815ca20 con 0x7f75b80cd9c0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b7fff640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80cd9c0 0x7f75b806f160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c460 tx=0x7f75a400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b57fa640 1 -- 172.21.15.79:0/3830552767 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4018070 con 0x7f75b80cd9c0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b57fa640 1 -- 172.21.15.79:0/3830552767 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75a4007520 con 0x7f75b80cd9c0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b815cd00 con 0x7f75b80cd9c0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.553+0000 7f75b57fa640 1 -- 172.21.15.79:0/3830552767 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a4004a20 con 0x7f75b80cd9c0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.554+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b815d2c0 con 0x7f75b80cd9c0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.554+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b80506c0 con 0x7f75b80cd9c0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.555+0000 7f75b57fa640 1 -- 172.21.15.79:0/3830552767 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75a401b070 con 0x7f75b80cd9c0 2024-04-26T19:51:19.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.555+0000 7f75b57fa640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f759c06c8f0 0x7f759c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:19.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.555+0000 7f75b57fa640 1 -- 172.21.15.79:0/3830552767 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f75a408b2a0 con 0x7f75b80cd9c0 2024-04-26T19:51:19.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.555+0000 7f75bcb5d640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f759c06c8f0 0x7f759c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:19.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.556+0000 7f75bcb5d640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f759c06c8f0 0x7f759c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f75ac000f60 tx=0x7f75ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:19.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.560+0000 7f75b57fa640 1 -- 172.21.15.79:0/3830552767 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f75a4058c80 con 0x7f75b80cd9c0 2024-04-26T19:51:19.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:19.803+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) -- 0x7f75b80b24c0 con 0x7f75b80cd9c0 2024-04-26T19:51:20.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.265+0000 7f75b57fa640 1 -- 172.21.15.79:0/3830552767 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v70) ==== 158+0+0 (secure 0 0 0) 0x7f75a408b0c0 con 0x7f75b80cd9c0 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f759c06c8f0 msgr2=0x7f759c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f759c06c8f0 0x7f759c06edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f75ac000f60 tx=0x7f75ac009040 comp rx=0 tx=0).stop 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80cd9c0 msgr2=0x7f75b806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80cd9c0 0x7f75b806f160 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f75a400c460 tx=0x7f75a400c920 comp rx=0 tx=0).stop 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 shutdown_connections 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f759c06c8f0 0x7f759c06edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f75b8156670 0x7f75b8145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f75b80cd9c0 0x7f75b806f160 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 --2- 172.21.15.79:0/3830552767 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f75b80b44f0 0x7f75b8147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 >> 172.21.15.79:0/3830552767 conn(0x7f75b80b39d0 msgr2=0x7f75b8150a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 shutdown_connections 2024-04-26T19:51:20.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.268+0000 7f75bedc1640 1 -- 172.21.15.79:0/3830552767 wait complete. 2024-04-26T19:51:20.280 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-26T19:51:20.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.379+0000 7f064d062640 1 Processor -- start 2024-04-26T19:51:20.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.379+0000 7f064d062640 1 -- start start 2024-04-26T19:51:20.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.380+0000 7f064d062640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f064815c1e0 0x7f0648073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.380+0000 7f064d062640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0648072620 con 0x7f0648072000 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.380+0000 7f064d062640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06480730d0 con 0x7f064806f920 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.380+0000 7f064d062640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f06480733b0 con 0x7f064815c1e0 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.380+0000 7f064759e640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0648072000 0x7f06480cdc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33254/0 (socket says 172.21.15.79:33254) 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.380+0000 7f064759e640 1 -- 172.21.15.79:0/1672620268 learned_addr learned my addr 172.21.15.79:0/1672620268 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.380+0000 7f0646d9d640 1 --2- 172.21.15.79:0/1672620268 >> v2:172.21.15.83:3300/0 conn(0x7f064815c1e0 0x7f0648073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.380+0000 7f0645d9b640 1 -- 172.21.15.79:0/1672620268 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2435294752 0 0) 0x7f0648072620 con 0x7f0648072000 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.380+0000 7f0645d9b640 1 -- 172.21.15.79:0/1672620268 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0630003520 con 0x7f0648072000 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.381+0000 7f0645d9b640 1 -- 172.21.15.79:0/1672620268 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3801635624 0 0) 0x7f0630003520 con 0x7f0648072000 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.381+0000 7f0646d9d640 1 -- 172.21.15.79:0/1672620268 >> v1:172.21.15.83:6789/0 conn(0x7f064806f920 legacy=0x7f06480718f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.381+0000 7f0646d9d640 1 -- 172.21.15.79:0/1672620268 >> v1:172.21.15.79:6789/0 conn(0x7f0648072000 legacy=0x7f06480cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.381+0000 7f0646d9d640 1 -- 172.21.15.79:0/1672620268 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0648073690 con 0x7f064815c1e0 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.381+0000 7f0646d9d640 1 --2- 172.21.15.79:0/1672620268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064815c1e0 0x7f0648073bb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0638009d90 tx=0x7f063802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0f0ffeef8cede1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.381+0000 7f0645d9b640 1 -- 172.21.15.79:0/1672620268 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640001670 con 0x7f064815c1e0 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.381+0000 7f0645d9b640 1 -- 172.21.15.79:0/1672620268 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0638007540 con 0x7f064815c1e0 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.382+0000 7f0645d9b640 1 -- 172.21.15.79:0/1672620268 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06380078f0 con 0x7f064815c1e0 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.382+0000 7f064d062640 1 -- 172.21.15.79:0/1672620268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064815c1e0 msgr2=0x7f0648073bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.382+0000 7f064d062640 1 --2- 172.21.15.79:0/1672620268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064815c1e0 0x7f0648073bb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0638009d90 tx=0x7f063802a560 comp rx=0 tx=0).stop 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.382+0000 7f064d062640 1 -- 172.21.15.79:0/1672620268 shutdown_connections 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.382+0000 7f064d062640 1 --2- 172.21.15.79:0/1672620268 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f064815c1e0 0x7f0648073bb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.382+0000 7f064d062640 1 -- 172.21.15.79:0/1672620268 >> 172.21.15.79:0/1672620268 conn(0x7f06480b2540 msgr2=0x7f06480b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.382+0000 7f064d062640 1 -- 172.21.15.79:0/1672620268 shutdown_connections 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.382+0000 7f064d062640 1 -- 172.21.15.79:0/1672620268 wait complete. 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.383+0000 7f064d062640 1 Processor -- start 2024-04-26T19:51:20.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.383+0000 7f064d062640 1 -- start start 2024-04-26T19:51:20.387 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064d062640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f064806f920 0x7f0648144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064d062640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648072000 0x7f06480ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064d062640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f064815c1e0 0x7f06480aac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064d062640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0648070180 con 0x7f064815c1e0 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064d062640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0648070000 con 0x7f0648072000 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064d062640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f06480ce3a0 con 0x7f064806f920 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064759e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f064815c1e0 0x7f06480aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064759e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f064815c1e0 0x7f06480aac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57054/0 (socket says 172.21.15.79:57054) 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064759e640 1 -- 172.21.15.79:0/2004908601 learned_addr learned my addr 172.21.15.79:0/2004908601 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f064659c640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648072000 0x7f06480ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.384+0000 7f0646d9d640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f064806f920 0x7f0648144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f064759e640 1 -- 172.21.15.79:0/2004908601 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f064806f920 msgr2=0x7f0648144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f064759e640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f064806f920 0x7f0648144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f064759e640 1 -- 172.21.15.79:0/2004908601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648072000 msgr2=0x7f06480ae3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f064759e640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648072000 0x7f06480ae3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f064759e640 1 -- 172.21.15.79:0/2004908601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06480aa4c0 con 0x7f064815c1e0 2024-04-26T19:51:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f064659c640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648072000 0x7f06480ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f064759e640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f064815c1e0 0x7f06480aac80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f064000c750 tx=0x7f064000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f062f7fe640 1 -- 172.21.15.79:0/2004908601 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640017070 con 0x7f064815c1e0 2024-04-26T19:51:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f062f7fe640 1 -- 172.21.15.79:0/2004908601 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0640005ad0 con 0x7f064815c1e0 2024-04-26T19:51:20.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06480aee40 con 0x7f064815c1e0 2024-04-26T19:51:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f062f7fe640 1 -- 172.21.15.79:0/2004908601 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640007070 con 0x7f064815c1e0 2024-04-26T19:51:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.385+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06480af400 con 0x7f064815c1e0 2024-04-26T19:51:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.386+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06480506c0 con 0x7f064815c1e0 2024-04-26T19:51:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.387+0000 7f062f7fe640 1 -- 172.21.15.79:0/2004908601 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f064001a070 con 0x7f064815c1e0 2024-04-26T19:51:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.387+0000 7f062f7fe640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f063406c8f0 0x7f063406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.387+0000 7f062f7fe640 1 -- 172.21.15.79:0/2004908601 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f064008b250 con 0x7f064815c1e0 2024-04-26T19:51:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.387+0000 7f0646d9d640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f063406c8f0 0x7f063406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.387+0000 7f0646d9d640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f063406c8f0 0x7f063406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0638034000 tx=0x7f063802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:20.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.391+0000 7f062f7fe640 1 -- 172.21.15.79:0/2004908601 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f0640058cc0 con 0x7f064815c1e0 2024-04-26T19:51:20.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.567+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0648072a60 con 0x7f064815c1e0 2024-04-26T19:51:20.571 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.571+0000 7f062f7fe640 1 -- 172.21.15.79:0/2004908601 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f064005cfc0 con 0x7f064815c1e0 2024-04-26T19:51:20.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f063406c8f0 msgr2=0x7f063406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.573 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f063406c8f0 0x7f063406edb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0638034000 tx=0x7f063802f000 comp rx=0 tx=0).stop 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f064815c1e0 msgr2=0x7f06480aac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f064815c1e0 0x7f06480aac80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f064000c750 tx=0x7f064000cc10 comp rx=0 tx=0).stop 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 shutdown_connections 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f063406c8f0 0x7f063406edb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f064815c1e0 0x7f06480aac80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0648072000 0x7f06480ae3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 --2- 172.21.15.79:0/2004908601 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f064806f920 0x7f0648144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 >> 172.21.15.79:0/2004908601 conn(0x7f06480b2540 msgr2=0x7f06480ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 shutdown_connections 2024-04-26T19:51:20.574 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.573+0000 7f064d062640 1 -- 172.21.15.79:0/2004908601 wait complete. 2024-04-26T19:51:20.584 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T19:51:20.585 DEBUG:teuthology.orchestra.run.smithi079:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T19:51:20.618 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T19:51:20.673 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T19:51:20.673 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:51:20.673 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:20.688 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:51:20.688 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:20.744 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:51:20.745 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:20.799 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:20.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.935+0000 7f9ce5c37640 1 Processor -- start 2024-04-26T19:51:20.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.935+0000 7f9ce5c37640 1 -- start start 2024-04-26T19:51:20.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9ce5c37640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce0156670 0x7f9ce015d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9ce5c37640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9ce00cd9c0 0x7f9ce00b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9ce5c37640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce015d670 con 0x7f9ce00735a0 2024-04-26T19:51:20.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9ce5c37640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce0073110 con 0x7f9ce00cd9c0 2024-04-26T19:51:20.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9ce5c37640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce00ce1a0 con 0x7f9ce0156670 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cdeffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9ce00cd9c0 0x7f9ce00b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cdffff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f9ce00735a0 0x7f9ce015fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33266/0 (socket says 172.21.15.79:33266) 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cdffff640 1 -- 172.21.15.79:0/862914495 learned_addr learned my addr 172.21.15.79:0/862914495 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cdf7fe640 1 --2- 172.21.15.79:0/862914495 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce0156670 0x7f9ce015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cdeffd640 1 -- 172.21.15.79:0/862914495 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce0156670 msgr2=0x7f9ce015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cdeffd640 1 --2- 172.21.15.79:0/862914495 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce0156670 0x7f9ce015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cdeffd640 1 -- 172.21.15.79:0/862914495 >> v1:172.21.15.79:6789/0 conn(0x7f9ce00735a0 legacy=0x7f9ce015fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cdeffd640 1 -- 172.21.15.79:0/862914495 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce00cdec0 con 0x7f9ce00cd9c0 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cde7fc640 1 -- 172.21.15.79:0/862914495 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 587537019 0 0) 0x7f9cd8001490 con 0x7f9ce00735a0 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.936+0000 7f9cdf7fe640 1 --2- 172.21.15.79:0/862914495 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce0156670 0x7f9ce015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9cdeffd640 1 --2- 172.21.15.79:0/862914495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00cd9c0 0x7f9ce00b3de0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00097a0 tx=0x7f9cd002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5ad2d7ec27423d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9cde7fc640 1 -- 172.21.15.79:0/862914495 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0037070 con 0x7f9ce00cd9c0 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9cde7fc640 1 -- 172.21.15.79:0/862914495 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd0032440 con 0x7f9ce00cd9c0 2024-04-26T19:51:20.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9cde7fc640 1 -- 172.21.15.79:0/862914495 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0032810 con 0x7f9ce00cd9c0 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9ce5c37640 1 -- 172.21.15.79:0/862914495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00cd9c0 msgr2=0x7f9ce00b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9ce5c37640 1 --2- 172.21.15.79:0/862914495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00cd9c0 0x7f9ce00b3de0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9cd00097a0 tx=0x7f9cd002a180 comp rx=0 tx=0).stop 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9ce5c37640 1 -- 172.21.15.79:0/862914495 shutdown_connections 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9ce5c37640 1 --2- 172.21.15.79:0/862914495 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00cd9c0 0x7f9ce00b3de0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9ce5c37640 1 --2- 172.21.15.79:0/862914495 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce0156670 0x7f9ce015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.937+0000 7f9ce5c37640 1 -- 172.21.15.79:0/862914495 >> 172.21.15.79:0/862914495 conn(0x7f9ce00b39d0 msgr2=0x7f9ce006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.938+0000 7f9ce5c37640 1 -- 172.21.15.79:0/862914495 shutdown_connections 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.938+0000 7f9ce5c37640 1 -- 172.21.15.79:0/862914495 wait complete. 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.939+0000 7f9ce5c37640 1 Processor -- start 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.939+0000 7f9ce5c37640 1 -- start start 2024-04-26T19:51:20.942 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.939+0000 7f9ce5c37640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00735a0 0x7f9ce0157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.939+0000 7f9ce5c37640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce00cd9c0 0x7f9ce0158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.939+0000 7f9ce5c37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce0156670 0x7f9ce0159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.939+0000 7f9ce5c37640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0156a50 con 0x7f9ce00735a0 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.939+0000 7f9ce5c37640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce0072da0 con 0x7f9ce0156670 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.939+0000 7f9ce5c37640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce0156bd0 con 0x7f9ce00cd9c0 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdf7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00735a0 0x7f9ce0157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdf7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00735a0 0x7f9ce0157730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57074/0 (socket says 172.21.15.79:57074) 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdf7fe640 1 -- 172.21.15.79:0/1095278402 learned_addr learned my addr 172.21.15.79:0/1095278402 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdf7fe640 1 -- 172.21.15.79:0/1095278402 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce00cd9c0 msgr2=0x7f9ce0158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdeffd640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce00cd9c0 0x7f9ce0158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdf7fe640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce00cd9c0 0x7f9ce0158c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdf7fe640 1 -- 172.21.15.79:0/1095278402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce0156670 msgr2=0x7f9ce0159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdffff640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce0156670 0x7f9ce0159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.943 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdf7fe640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce0156670 0x7f9ce0159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:20.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdf7fe640 1 -- 172.21.15.79:0/1095278402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce0158140 con 0x7f9ce00735a0 2024-04-26T19:51:20.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cdf7fe640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00735a0 0x7f9ce0157730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400d7d0 tx=0x7f9cd400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:20.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cc7fff640 1 -- 172.21.15.79:0/1095278402 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4004030 con 0x7f9ce00735a0 2024-04-26T19:51:20.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cc7fff640 1 -- 172.21.15.79:0/1095278402 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd40041d0 con 0x7f9ce00735a0 2024-04-26T19:51:20.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9cc7fff640 1 -- 172.21.15.79:0/1095278402 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd40096a0 con 0x7f9ce00735a0 2024-04-26T19:51:20.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce0158420 con 0x7f9ce00735a0 2024-04-26T19:51:20.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.940+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce01521c0 con 0x7f9ce00735a0 2024-04-26T19:51:20.944 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.941+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce00506c0 con 0x7f9ce00735a0 2024-04-26T19:51:20.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.942+0000 7f9cc7fff640 1 -- 172.21.15.79:0/1095278402 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cd4009840 con 0x7f9ce00735a0 2024-04-26T19:51:20.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.942+0000 7f9cc7fff640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ccc06c920 0x7f9ccc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:20.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.942+0000 7f9cdeffd640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ccc06c920 0x7f9ccc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:20.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.942+0000 7f9cc7fff640 1 -- 172.21.15.79:0/1095278402 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9cd408c240 con 0x7f9ce00735a0 2024-04-26T19:51:20.945 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.943+0000 7f9cdeffd640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ccc06c920 0x7f9ccc06ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0034000 tx=0x7f9cd002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:20.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:20.947+0000 7f9cc7fff640 1 -- 172.21.15.79:0/1095278402 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9cd405a280 con 0x7f9ce00735a0 2024-04-26T19:51:21.122 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.121+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ce015dcd0 con 0x7f9ce00735a0 2024-04-26T19:51:21.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.122+0000 7f9cc7fff640 1 -- 172.21.15.79:0/1095278402 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7f9cd4059b80 con 0x7f9ce00735a0 2024-04-26T19:51:21.123 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:21.123 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.124+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ccc06c920 msgr2=0x7f9ccc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.124+0000 7f9ce5c37640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ccc06c920 0x7f9ccc06ede0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0034000 tx=0x7f9cd002e000 comp rx=0 tx=0).stop 2024-04-26T19:51:21.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.124+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00735a0 msgr2=0x7f9ce0157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.124+0000 7f9ce5c37640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00735a0 0x7f9ce0157730 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400d7d0 tx=0x7f9cd400dc90 comp rx=0 tx=0).stop 2024-04-26T19:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.125+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 shutdown_connections 2024-04-26T19:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.125+0000 7f9ce5c37640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9ccc06c920 0x7f9ccc06ede0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.125+0000 7f9ce5c37640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9ce0156670 0x7f9ce0159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.125+0000 7f9ce5c37640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9ce00cd9c0 0x7f9ce0158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.125+0000 7f9ce5c37640 1 --2- 172.21.15.79:0/1095278402 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9ce00735a0 0x7f9ce0157730 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.125+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 >> 172.21.15.79:0/1095278402 conn(0x7f9ce00b39d0 msgr2=0x7f9ce015e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.125+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 shutdown_connections 2024-04-26T19:51:21.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.125+0000 7f9ce5c37640 1 -- 172.21.15.79:0/1095278402 wait complete. 2024-04-26T19:51:21.136 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:21.137 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:21.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.237+0000 7f7638d46640 1 Processor -- start 2024-04-26T19:51:21.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.237+0000 7f7638d46640 1 -- start start 2024-04-26T19:51:21.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f7638d46640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7634163620 0x7f7634163a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f7638d46640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7634072930 0x7f763415fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f7638d46640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76341603b0 con 0x7f763415f960 2024-04-26T19:51:21.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f7638d46640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f76340b0280 con 0x7f7634163620 2024-04-26T19:51:21.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f7638d46640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7634160690 con 0x7f7634072930 2024-04-26T19:51:21.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f763259c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7634163620 0x7f7634163a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f7631d9b640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f763415f960 0x7f7634072220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33280/0 (socket says 172.21.15.79:33280) 2024-04-26T19:51:21.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f7631d9b640 1 -- 172.21.15.79:0/2184788658 learned_addr learned my addr 172.21.15.79:0/2184788658 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f7632d9d640 1 --2- 172.21.15.79:0/2184788658 >> v2:172.21.15.83:3300/0 conn(0x7f7634072930 0x7f763415fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f763259c640 1 -- 172.21.15.79:0/2184788658 >> v2:172.21.15.83:3300/0 conn(0x7f7634072930 msgr2=0x7f763415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f763259c640 1 --2- 172.21.15.79:0/2184788658 >> v2:172.21.15.83:3300/0 conn(0x7f7634072930 0x7f763415fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f763259c640 1 -- 172.21.15.79:0/2184788658 >> v1:172.21.15.79:6789/0 conn(0x7f763415f960 legacy=0x7f7634072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f763259c640 1 -- 172.21.15.79:0/2184788658 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76340aea70 con 0x7f7634163620 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.238+0000 7f763159a640 1 -- 172.21.15.79:0/2184788658 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 486909034 0 0) 0x7f76341603b0 con 0x7f763415f960 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f763259c640 1 --2- 172.21.15.79:0/2184788658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 0x7f7634163a00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7634163f40 tx=0x7f762402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f4be7c603a2548 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f763159a640 1 -- 172.21.15.79:0/2184788658 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624007740 con 0x7f7634163620 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f763159a640 1 -- 172.21.15.79:0/2184788658 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76240078e0 con 0x7f7634163620 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f763159a640 1 -- 172.21.15.79:0/2184788658 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7624007cb0 con 0x7f7634163620 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f7638d46640 1 -- 172.21.15.79:0/2184788658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 msgr2=0x7f7634163a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f7638d46640 1 --2- 172.21.15.79:0/2184788658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 0x7f7634163a00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7634163f40 tx=0x7f762402a5f0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f7638d46640 1 -- 172.21.15.79:0/2184788658 shutdown_connections 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f7638d46640 1 --2- 172.21.15.79:0/2184788658 >> v2:172.21.15.83:3300/0 conn(0x7f7634072930 0x7f763415fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f7638d46640 1 --2- 172.21.15.79:0/2184788658 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 0x7f7634163a00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f7638d46640 1 -- 172.21.15.79:0/2184788658 >> 172.21.15.79:0/2184788658 conn(0x7f76340b2e30 msgr2=0x7f76340b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:21.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f7638d46640 1 -- 172.21.15.79:0/2184788658 shutdown_connections 2024-04-26T19:51:21.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.239+0000 7f7638d46640 1 -- 172.21.15.79:0/2184788658 wait complete. 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.241+0000 7f7638d46640 1 Processor -- start 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.241+0000 7f7638d46640 1 -- start start 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.241+0000 7f7638d46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7634072930 0x7f76340d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.241+0000 7f7638d46640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f763415f960 0x7f763415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.241+0000 7f7638d46640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 0x7f7634144550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.241+0000 7f7638d46640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f763406f030 con 0x7f7634163620 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.241+0000 7f7638d46640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f763406eeb0 con 0x7f7634072930 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.241+0000 7f7638d46640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f76340af270 con 0x7f763415f960 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.241+0000 7f7632d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 0x7f7634144550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7632d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 0x7f7634144550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57098/0 (socket says 172.21.15.79:57098) 2024-04-26T19:51:21.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7632d9d640 1 -- 172.21.15.79:0/1700079003 learned_addr learned my addr 172.21.15.79:0/1700079003 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f763259c640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7634072930 0x7f76340d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7631d9b640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f763415f960 0x7f763415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7632d9d640 1 -- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f763415f960 msgr2=0x7f763415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7632d9d640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f763415f960 0x7f763415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7632d9d640 1 -- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7634072930 msgr2=0x7f76340d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7632d9d640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7634072930 0x7f76340d0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7632d9d640 1 -- 172.21.15.79:0/1700079003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763415aa50 con 0x7f7634163620 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f763259c640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7634072930 0x7f76340d0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7631d9b640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f763415f960 0x7f763415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7632d9d640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 0x7f7634144550 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c7c0 tx=0x7f762c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7622ffd640 1 -- 172.21.15.79:0/1700079003 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c015070 con 0x7f7634163620 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7622ffd640 1 -- 172.21.15.79:0/1700079003 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f762c004220 con 0x7f7634163620 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763415ad30 con 0x7f7634163620 2024-04-26T19:51:21.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7622ffd640 1 -- 172.21.15.79:0/1700079003 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f762c007620 con 0x7f7634163620 2024-04-26T19:51:21.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.242+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763415b2f0 con 0x7f7634163620 2024-04-26T19:51:21.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.243+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76340506c0 con 0x7f7634163620 2024-04-26T19:51:21.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.244+0000 7f7622ffd640 1 -- 172.21.15.79:0/1700079003 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f762c0077c0 con 0x7f7634163620 2024-04-26T19:51:21.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.244+0000 7f7622ffd640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f761406c8f0 0x7f761406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.244+0000 7f763259c640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f761406c8f0 0x7f761406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.244+0000 7f7622ffd640 1 -- 172.21.15.79:0/1700079003 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f762c08bdd0 con 0x7f7634163620 2024-04-26T19:51:21.247 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.245+0000 7f763259c640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f761406c8f0 0x7f761406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7624034000 tx=0x7f7624033040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:21.249 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.248+0000 7f7622ffd640 1 -- 172.21.15.79:0/1700079003 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f762c059e30 con 0x7f7634163620 2024-04-26T19:51:21.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.424+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f76340d1030 con 0x7f7634163620 2024-04-26T19:51:21.427 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.424+0000 7f7622ffd640 1 -- 172.21.15.79:0/1700079003 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f76340d1030 con 0x7f7634163620 2024-04-26T19:51:21.427 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f761406c8f0 msgr2=0x7f761406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f761406c8f0 0x7f761406edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7624034000 tx=0x7f7624033040 comp rx=0 tx=0).stop 2024-04-26T19:51:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 msgr2=0x7f7634144550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 0x7f7634144550 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f762c00c7c0 tx=0x7f762c00cc80 comp rx=0 tx=0).stop 2024-04-26T19:51:21.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 shutdown_connections 2024-04-26T19:51:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f761406c8f0 0x7f761406edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7634163620 0x7f7634144550 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f763415f960 0x7f763415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 --2- 172.21.15.79:0/1700079003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7634072930 0x7f76340d0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 >> 172.21.15.79:0/1700079003 conn(0x7f76340b2e30 msgr2=0x7f76340b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:21.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.426+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 shutdown_connections 2024-04-26T19:51:21.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.427+0000 7f7638d46640 1 -- 172.21.15.79:0/1700079003 wait complete. 2024-04-26T19:51:21.438 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:21.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.538+0000 7fe3fe16a640 1 Processor -- start 2024-04-26T19:51:21.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.538+0000 7fe3fe16a640 1 -- start start 2024-04-26T19:51:21.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.539+0000 7fe3fe16a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f8148f30 0x7fe3f80b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.539+0000 7fe3fe16a640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f80b7e60 con 0x7fe3f80b7620 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.539+0000 7fe3fe16a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f80b8140 con 0x7fe3f806c350 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.539+0000 7fe3fe16a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f80b8420 con 0x7fe3f8148f30 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.539+0000 7fe3f6ffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe3f80b7620 0x7fe3f80b8f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33290/0 (socket says 172.21.15.79:33290) 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.539+0000 7fe3f6ffd640 1 -- 172.21.15.79:0/4224904968 learned_addr learned my addr 172.21.15.79:0/4224904968 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4059912336 0 0) 0x7fe3f80b7e60 con 0x7fe3f80b7620 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3e0003520 con 0x7fe3f80b7620 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f77fe640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f8148f30 0x7fe3f80b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 96227233 0 0) 0x7fe3e0003520 con 0x7fe3f80b7620 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f8148f30 msgr2=0x7fe3f80b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f8148f30 0x7fe3f80b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 >> v1:172.21.15.83:6789/0 conn(0x7fe3f806c350 legacy=0x7fe3f815fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f80b8700 con 0x7fe3f80b7620 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fe3ec001670 con 0x7fe3f80b7620 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 >> v1:172.21.15.79:6789/0 conn(0x7fe3f80b7620 legacy=0x7fe3f80b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e00039e0 0x7fe3e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f8148f30 0x7fe3e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f80b8420 con 0x7fe3e0004800 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e000ba70 con 0x7fe3e00039e0 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e000bc70 con 0x7fe3f8148f30 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f6ffd640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f77fe640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e00039e0 0x7fe3e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f7fff640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f8148f30 0x7fe3e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f6ffd640 1 -- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f8148f30 msgr2=0x7fe3e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:21.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f6ffd640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f8148f30 0x7fe3e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f6ffd640 1 -- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e00039e0 msgr2=0x7fe3e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:21.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f6ffd640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e00039e0 0x7fe3e0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.540+0000 7fe3f6ffd640 1 -- 172.21.15.79:0/4224904968 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e0003520 con 0x7fe3e0004800 2024-04-26T19:51:21.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.541+0000 7fe3f6ffd640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3e0004c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec001eb0 tx=0x7fe3e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7376e6cdb87be03b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:21.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.541+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec007d30 con 0x7fe3e0004800 2024-04-26T19:51:21.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.541+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ec005af0 con 0x7fe3e0004800 2024-04-26T19:51:21.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.541+0000 7fe3f67fc640 1 -- 172.21.15.79:0/4224904968 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec007070 con 0x7fe3e0004800 2024-04-26T19:51:21.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.541+0000 7fe3fe16a640 1 -- 172.21.15.79:0/4224904968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 msgr2=0x7fe3e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.541+0000 7fe3fe16a640 1 --2- 172.21.15.79:0/4224904968 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3e0004c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec001eb0 tx=0x7fe3e0005830 comp rx=0 tx=0).stop 2024-04-26T19:51:21.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.542+0000 7fe3f7fff640 1 -- 172.21.15.79:0/4224904968 reap_dead start 2024-04-26T19:51:21.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.542+0000 7fe3fe16a640 1 -- 172.21.15.79:0/4224904968 shutdown_connections 2024-04-26T19:51:21.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.542+0000 7fe3fe16a640 1 -- 172.21.15.79:0/4224904968 >> 172.21.15.79:0/4224904968 conn(0x7fe3f8073b40 msgr2=0x7fe3f8073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:21.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.542+0000 7fe3fe16a640 1 -- 172.21.15.79:0/4224904968 shutdown_connections 2024-04-26T19:51:21.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.542+0000 7fe3fe16a640 1 -- 172.21.15.79:0/4224904968 wait complete. 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.543+0000 7fe3fe16a640 1 Processor -- start 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.543+0000 7fe3fe16a640 1 -- start start 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3fe16a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3f81f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3fe16a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f815eb20 0x7fe3f8161df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3fe16a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f815fdc0 0x7fe3f8160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3fe16a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f80b7be0 con 0x7fe3e0004800 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3fe16a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f80b7a60 con 0x7fe3f815eb20 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3fe16a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f80b6630 con 0x7fe3f815fdc0 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f77fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3f81f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f77fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3f81f9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57106/0 (socket says 172.21.15.79:57106) 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f77fe640 1 -- 172.21.15.79:0/547515386 learned_addr learned my addr 172.21.15.79:0/547515386 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:21.549 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f6ffd640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f815eb20 0x7fe3f8161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f77fe640 1 -- 172.21.15.79:0/547515386 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f815fdc0 msgr2=0x7fe3f8160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f77fe640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f815fdc0 0x7fe3f8160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f77fe640 1 -- 172.21.15.79:0/547515386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f815eb20 msgr2=0x7fe3f8161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f77fe640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f815eb20 0x7fe3f8161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f77fe640 1 -- 172.21.15.79:0/547515386 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f813df20 con 0x7fe3e0004800 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3f77fe640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3f81f9170 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80119a0 tx=0x7fe3e8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3dffff640 1 -- 172.21.15.79:0/547515386 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800fa90 con 0x7fe3e0004800 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3dffff640 1 -- 172.21.15.79:0/547515386 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e800fc30 con 0x7fe3e0004800 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f8162690 con 0x7fe3e0004800 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3dffff640 1 -- 172.21.15.79:0/547515386 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8014440 con 0x7fe3e0004800 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.544+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f8162c50 con 0x7fe3e0004800 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.545+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f80506c0 con 0x7fe3e0004800 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.546+0000 7fe3dffff640 1 -- 172.21.15.79:0/547515386 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3e8012040 con 0x7fe3e0004800 2024-04-26T19:51:21.550 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.546+0000 7fe3dffff640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3e406c920 0x7fe3e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.546+0000 7fe3dffff640 1 -- 172.21.15.79:0/547515386 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe3e808c610 con 0x7fe3e0004800 2024-04-26T19:51:21.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.546+0000 7fe3f6ffd640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3e406c920 0x7fe3e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.547+0000 7fe3f6ffd640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3e406c920 0x7fe3e406ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec032000 tx=0x7fe3ec02d000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:21.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.551+0000 7fe3dffff640 1 -- 172.21.15.79:0/547515386 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3e8019030 con 0x7fe3e0004800 2024-04-26T19:51:21.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.725+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe3f8162e90 con 0x7fe3e0004800 2024-04-26T19:51:21.725 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.725+0000 7fe3dffff640 1 -- 172.21.15.79:0/547515386 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fe3e805a640 con 0x7fe3e0004800 2024-04-26T19:51:21.726 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3e406c920 msgr2=0x7fe3e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3e406c920 0x7fe3e406ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec032000 tx=0x7fe3ec02d000 comp rx=0 tx=0).stop 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 msgr2=0x7fe3f81f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3f81f9170 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe3e80119a0 tx=0x7fe3e8011e60 comp rx=0 tx=0).stop 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 shutdown_connections 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3e406c920 0x7fe3e406ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3f815fdc0 0x7fe3f8160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3f815eb20 0x7fe3f8161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 --2- 172.21.15.79:0/547515386 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e0004800 0x7fe3f81f9170 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 >> 172.21.15.79:0/547515386 conn(0x7fe3f8073b40 msgr2=0x7fe3f80b57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 shutdown_connections 2024-04-26T19:51:21.728 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.727+0000 7fe3fe16a640 1 -- 172.21.15.79:0/547515386 wait complete. 2024-04-26T19:51:21.739 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:21.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.838+0000 7f4f16f6a640 1 Processor -- start 2024-04-26T19:51:21.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.838+0000 7f4f16f6a640 1 -- start start 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f16f6a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f10161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f16f6a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f4f1006df80 0x7f4f10073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f16f6a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4f10073f60 0x7f4f1015d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f16f6a640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f4f10161bd0 con 0x7f4f10073f60 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f16f6a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f4f1006e540 con 0x7f4f1006df80 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f16f6a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1006e820 con 0x7f4f10146bd0 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f15507640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4f10073f60 0x7f4f1015d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f15507640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4f10073f60 0x7f4f1015d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57118/0 (socket says 172.21.15.79:57118) 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f15507640 1 -- 172.21.15.79:0/221275473 learned_addr learned my addr 172.21.15.79:0/221275473 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f14d06640 1 --2- 172.21.15.79:0/221275473 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f10161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.839+0000 7f4f0ffff640 1 --2- 172.21.15.79:0/221275473 >> v2:172.21.15.83:3300/0 conn(0x7f4f1006df80 0x7f4f10073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f15507640 1 -- 172.21.15.79:0/221275473 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 msgr2=0x7f4f10161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f15507640 1 --2- 172.21.15.79:0/221275473 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f10161690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f15507640 1 -- 172.21.15.79:0/221275473 >> v2:172.21.15.83:3300/0 conn(0x7f4f1006df80 msgr2=0x7f4f10073a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f15507640 1 --2- 172.21.15.79:0/221275473 >> v2:172.21.15.83:3300/0 conn(0x7f4f1006df80 0x7f4f10073a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f15507640 1 -- 172.21.15.79:0/221275473 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1006eb00 con 0x7f4f10073f60 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f14d06640 1 --2- 172.21.15.79:0/221275473 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f10161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f0ffff640 1 --2- 172.21.15.79:0/221275473 >> v2:172.21.15.83:3300/0 conn(0x7f4f1006df80 0x7f4f10073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f15507640 1 --2- 172.21.15.79:0/221275473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f10073f60 0x7f4f1015d6b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4f100b3320 tx=0x7f4f0802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a8daf112db684b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f0f7fe640 1 -- 172.21.15.79:0/221275473 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f08036070 con 0x7f4f10073f60 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f0f7fe640 1 -- 172.21.15.79:0/221275473 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f08031600 con 0x7f4f10073f60 2024-04-26T19:51:21.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.840+0000 7f4f0f7fe640 1 -- 172.21.15.79:0/221275473 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080319d0 con 0x7f4f10073f60 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.841+0000 7f4f16f6a640 1 -- 172.21.15.79:0/221275473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f10073f60 msgr2=0x7f4f1015d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.841+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/221275473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f10073f60 0x7f4f1015d6b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4f100b3320 tx=0x7f4f0802a6a0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.841+0000 7f4f16f6a640 1 -- 172.21.15.79:0/221275473 shutdown_connections 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.841+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/221275473 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f10073f60 0x7f4f1015d6b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.841+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/221275473 >> v2:172.21.15.83:3300/0 conn(0x7f4f1006df80 0x7f4f10073a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.841+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/221275473 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f10161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.841+0000 7f4f16f6a640 1 -- 172.21.15.79:0/221275473 >> 172.21.15.79:0/221275473 conn(0x7f4f100b2540 msgr2=0x7f4f100b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.841+0000 7f4f16f6a640 1 -- 172.21.15.79:0/221275473 shutdown_connections 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.841+0000 7f4f16f6a640 1 -- 172.21.15.79:0/221275473 wait complete. 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.842+0000 7f4f16f6a640 1 Processor -- start 2024-04-26T19:51:21.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.842+0000 7f4f16f6a640 1 -- start start 2024-04-26T19:51:21.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f16f6a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f1006df80 0x7f4f100af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f16f6a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f10073f60 0x7f4f10144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f16f6a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f1015c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f16f6a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1015dbf0 con 0x7f4f1006df80 2024-04-26T19:51:21.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f16f6a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1015e850 con 0x7f4f10073f60 2024-04-26T19:51:21.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f16f6a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4f1015dd70 con 0x7f4f10146bd0 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f14d06640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f1006df80 0x7f4f100af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f14d06640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f1006df80 0x7f4f100af0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57126/0 (socket says 172.21.15.79:57126) 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f14d06640 1 -- 172.21.15.79:0/2726392583 learned_addr learned my addr 172.21.15.79:0/2726392583 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f15507640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f1015c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f0ffff640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f10073f60 0x7f4f10144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f14d06640 1 -- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 msgr2=0x7f4f1015c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f14d06640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f1015c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f14d06640 1 -- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f10073f60 msgr2=0x7f4f10144d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f14d06640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f10073f60 0x7f4f10144d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f14d06640 1 -- 172.21.15.79:0/2726392583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f100718b0 con 0x7f4f1006df80 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f0ffff640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f10073f60 0x7f4f10144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:21.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.843+0000 7f4f15507640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f1015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.844+0000 7f4f14d06640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f1006df80 0x7f4f100af0c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000d7d0 tx=0x7f4f0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.844+0000 7f4f0d7fa640 1 -- 172.21.15.79:0/2726392583 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00004030 con 0x7f4f1006df80 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.844+0000 7f4f0d7fa640 1 -- 172.21.15.79:0/2726392583 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f000041d0 con 0x7f4f1006df80 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.844+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f10071b90 con 0x7f4f1006df80 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.844+0000 7f4f0d7fa640 1 -- 172.21.15.79:0/2726392583 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f000096a0 con 0x7f4f1006df80 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.844+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f10072430 con 0x7f4f1006df80 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.845+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f100506c0 con 0x7f4f1006df80 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.845+0000 7f4f0d7fa640 1 -- 172.21.15.79:0/2726392583 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4f00009840 con 0x7f4f1006df80 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.846+0000 7f4f0d7fa640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.846+0000 7f4f0ffff640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.846+0000 7f4f0d7fa640 1 -- 172.21.15.79:0/2726392583 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4f0008c250 con 0x7f4f1006df80 2024-04-26T19:51:21.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.846+0000 7f4f0ffff640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f1013f850 tx=0x7f4f04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:21.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:21.850+0000 7f4f0d7fa640 1 -- 172.21.15.79:0/2726392583 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4f0005a280 con 0x7f4f1006df80 2024-04-26T19:51:22.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.026+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f10071d60 con 0x7f4f1006df80 2024-04-26T19:51:22.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.026+0000 7f4f0d7fa640 1 -- 172.21.15.79:0/2726392583 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f4f00059b80 con 0x7f4f1006df80 2024-04-26T19:51:22.027 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 msgr2=0x7f4efc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4f1013f850 tx=0x7f4f04008040 comp rx=0 tx=0).stop 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f1006df80 msgr2=0x7f4f100af0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f1006df80 0x7f4f100af0c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000d7d0 tx=0x7f4f0000dc90 comp rx=0 tx=0).stop 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 shutdown_connections 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4efc06c8f0 0x7f4efc06edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f10146bd0 0x7f4f1015c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f10073f60 0x7f4f10144d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 --2- 172.21.15.79:0/2726392583 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f1006df80 0x7f4f100af0c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 >> 172.21.15.79:0/2726392583 conn(0x7f4f100b2540 msgr2=0x7f4f100b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 shutdown_connections 2024-04-26T19:51:22.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.028+0000 7f4f16f6a640 1 -- 172.21.15.79:0/2726392583 wait complete. 2024-04-26T19:51:22.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:22.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.138+0000 7f5409933640 1 Processor -- start 2024-04-26T19:51:22.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.138+0000 7f5409933640 1 -- start start 2024-04-26T19:51:22.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5409933640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f54041598c0 0x7f54040b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5409933640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 0x7f54040b9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5409933640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f540406f210 0x7f540406f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5409933640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f540406fbc0 con 0x7f54041598c0 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5409933640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f54040d5100 con 0x7f540406f210 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5409933640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f54040ba6f0 con 0x7f5404160610 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5402ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f54041598c0 0x7f54040b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5402ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f54041598c0 0x7f54040b9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57128/0 (socket says 172.21.15.79:57128) 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5402ffd640 1 -- 172.21.15.79:0/3442820497 learned_addr learned my addr 172.21.15.79:0/3442820497 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5402ffd640 1 -- 172.21.15.79:0/3442820497 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 msgr2=0x7f54040b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5402ffd640 1 --2- 172.21.15.79:0/3442820497 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 0x7f54040b9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f54037fe640 1 --2- 172.21.15.79:0/3442820497 >> v2:172.21.15.83:3300/0 conn(0x7f540406f210 0x7f540406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5402ffd640 1 -- 172.21.15.79:0/3442820497 >> v2:172.21.15.83:3300/0 conn(0x7f540406f210 msgr2=0x7f540406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5402ffd640 1 --2- 172.21.15.79:0/3442820497 >> v2:172.21.15.83:3300/0 conn(0x7f540406f210 0x7f540406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f5402ffd640 1 -- 172.21.15.79:0/3442820497 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54040ba410 con 0x7f54041598c0 2024-04-26T19:51:22.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.139+0000 7f54037fe640 1 --2- 172.21.15.79:0/3442820497 >> v2:172.21.15.83:3300/0 conn(0x7f540406f210 0x7f540406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:22.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.140+0000 7f5402ffd640 1 --2- 172.21.15.79:0/3442820497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54041598c0 0x7f54040b9140 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f540406dd60 tx=0x7f53f402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef97d6d174d514f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:22.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.140+0000 7f54027fc640 1 -- 172.21.15.79:0/3442820497 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4037070 con 0x7f54041598c0 2024-04-26T19:51:22.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.140+0000 7f54027fc640 1 -- 172.21.15.79:0/3442820497 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f4032440 con 0x7f54041598c0 2024-04-26T19:51:22.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.140+0000 7f54027fc640 1 -- 172.21.15.79:0/3442820497 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4032810 con 0x7f54041598c0 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.141+0000 7f5409933640 1 -- 172.21.15.79:0/3442820497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54041598c0 msgr2=0x7f54040b9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.141+0000 7f5409933640 1 --2- 172.21.15.79:0/3442820497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54041598c0 0x7f54040b9140 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f540406dd60 tx=0x7f53f402a140 comp rx=0 tx=0).stop 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.141+0000 7f5409933640 1 -- 172.21.15.79:0/3442820497 shutdown_connections 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.141+0000 7f5409933640 1 --2- 172.21.15.79:0/3442820497 >> v2:172.21.15.83:3300/0 conn(0x7f540406f210 0x7f540406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.141+0000 7f5409933640 1 --2- 172.21.15.79:0/3442820497 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 0x7f54040b9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.141+0000 7f5409933640 1 --2- 172.21.15.79:0/3442820497 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54041598c0 0x7f54040b9140 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.141+0000 7f5409933640 1 -- 172.21.15.79:0/3442820497 >> 172.21.15.79:0/3442820497 conn(0x7f54040b4930 msgr2=0x7f54040b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.141+0000 7f5409933640 1 -- 172.21.15.79:0/3442820497 shutdown_connections 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.141+0000 7f5409933640 1 -- 172.21.15.79:0/3442820497 wait complete. 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.142+0000 7f5409933640 1 Processor -- start 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.142+0000 7f5409933640 1 -- start start 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5409933640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f540406f210 0x7f54040b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5409933640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54041598c0 0x7f540415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5409933640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 0x7f54040b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5409933640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f54040ba070 con 0x7f540406f210 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5409933640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f54040b8940 con 0x7f54041598c0 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5409933640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f54040ba1f0 con 0x7f5404160610 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5402ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f540406f210 0x7f54040b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5402ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f540406f210 0x7f54040b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57138/0 (socket says 172.21.15.79:57138) 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5402ffd640 1 -- 172.21.15.79:0/4022082562 learned_addr learned my addr 172.21.15.79:0/4022082562 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f54037fe640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 0x7f54040b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f53fbfff640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54041598c0 0x7f540415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5402ffd640 1 -- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 msgr2=0x7f54040b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5402ffd640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 0x7f54040b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5402ffd640 1 -- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54041598c0 msgr2=0x7f540415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5402ffd640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54041598c0 0x7f540415ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5402ffd640 1 -- 172.21.15.79:0/4022082562 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540415b820 con 0x7f540406f210 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f54037fe640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 0x7f54040b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f53fbfff640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54041598c0 0x7f540415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5402ffd640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f540406f210 0x7f54040b3110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f53f4000c00 tx=0x7f53f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5400ff9640 1 -- 172.21.15.79:0/4022082562 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4041020 con 0x7f540406f210 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5400ff9640 1 -- 172.21.15.79:0/4022082562 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f4030970 con 0x7f540406f210 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5400ff9640 1 -- 172.21.15.79:0/4022082562 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f4037040 con 0x7f540406f210 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540415bb00 con 0x7f540406f210 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.143+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540415c0c0 con 0x7f540406f210 2024-04-26T19:51:22.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.144+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54040506c0 con 0x7f540406f210 2024-04-26T19:51:22.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.145+0000 7f5400ff9640 1 -- 172.21.15.79:0/4022082562 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53f4044020 con 0x7f540406f210 2024-04-26T19:51:22.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.145+0000 7f5400ff9640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f53e406c920 0x7f53e406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.145+0000 7f53fbfff640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f53e406c920 0x7f53e406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.145+0000 7f5400ff9640 1 -- 172.21.15.79:0/4022082562 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f53f40ac990 con 0x7f540406f210 2024-04-26T19:51:22.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.146+0000 7f53fbfff640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f53e406c920 0x7f53e406ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006fd0 tx=0x7f53ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:22.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.150+0000 7f5400ff9640 1 -- 172.21.15.79:0/4022082562 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f53f404b030 con 0x7f540406f210 2024-04-26T19:51:22.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.327+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5404076060 con 0x7f540406f210 2024-04-26T19:51:22.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.327+0000 7f5400ff9640 1 -- 172.21.15.79:0/4022082562 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f53f407a9c0 con 0x7f540406f210 2024-04-26T19:51:22.328 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:22.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f53e406c920 msgr2=0x7f53e406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f53e406c920 0x7f53e406ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f53ec006fd0 tx=0x7f53ec008040 comp rx=0 tx=0).stop 2024-04-26T19:51:22.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f540406f210 msgr2=0x7f54040b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f540406f210 0x7f54040b3110 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f53f4000c00 tx=0x7f53f40046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 shutdown_connections 2024-04-26T19:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f53e406c920 0x7f53e406ede0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5404160610 0x7f54040b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54041598c0 0x7f540415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 --2- 172.21.15.79:0/4022082562 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f540406f210 0x7f54040b3110 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.329+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 >> 172.21.15.79:0/4022082562 conn(0x7f54040b4930 msgr2=0x7f54041623a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.330+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 shutdown_connections 2024-04-26T19:51:22.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.330+0000 7f5409933640 1 -- 172.21.15.79:0/4022082562 wait complete. 2024-04-26T19:51:22.341 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T19:51:22.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.438+0000 7f2e47293640 1 Processor -- start 2024-04-26T19:51:22.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.438+0000 7f2e47293640 1 -- start start 2024-04-26T19:51:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.438+0000 7f2e47293640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e400b64f0 0x7f2e40070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.438+0000 7f2e47293640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2e4006e800 0x7f2e400b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.438+0000 7f2e47293640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4015f810 con 0x7f2e4015ffa0 2024-04-26T19:51:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.438+0000 7f2e47293640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f2e4015fb10 con 0x7f2e4006e800 2024-04-26T19:51:22.440 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.438+0000 7f2e47293640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2e4006f7a0 con 0x7f2e400b64f0 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e4482e640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2e4006e800 0x7f2e400b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e45830640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f2e4015ffa0 0x7f2e40160a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33292/0 (socket says 172.21.15.79:33292) 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e45830640 1 -- 172.21.15.79:0/956278987 learned_addr learned my addr 172.21.15.79:0/956278987 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e4482e640 1 -- 172.21.15.79:0/956278987 >> v2:172.21.15.83:3300/0 conn(0x7f2e400b64f0 msgr2=0x7f2e40070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e4502f640 1 --2- 172.21.15.79:0/956278987 >> v2:172.21.15.83:3300/0 conn(0x7f2e400b64f0 0x7f2e40070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e4482e640 1 --2- 172.21.15.79:0/956278987 >> v2:172.21.15.83:3300/0 conn(0x7f2e400b64f0 0x7f2e40070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e4482e640 1 -- 172.21.15.79:0/956278987 >> v1:172.21.15.79:6789/0 conn(0x7f2e4015ffa0 legacy=0x7f2e40160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e37fff640 1 -- 172.21.15.79:0/956278987 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 264960490 0 0) 0x7f2e4015f810 con 0x7f2e4015ffa0 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e4482e640 1 -- 172.21.15.79:0/956278987 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4006ec60 con 0x7f2e4006e800 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.439+0000 7f2e4502f640 1 --2- 172.21.15.79:0/956278987 >> v2:172.21.15.83:3300/0 conn(0x7f2e400b64f0 0x7f2e40070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e4482e640 1 --2- 172.21.15.79:0/956278987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 0x7f2e400b68d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2e4006d3b0 tx=0x7f2e3c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4870b3b31c89dba5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e37fff640 1 -- 172.21.15.79:0/956278987 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c037070 con 0x7f2e4006e800 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e37fff640 1 -- 172.21.15.79:0/956278987 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3c032440 con 0x7f2e4006e800 2024-04-26T19:51:22.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e37fff640 1 -- 172.21.15.79:0/956278987 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c032810 con 0x7f2e4006e800 2024-04-26T19:51:22.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e47293640 1 -- 172.21.15.79:0/956278987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 msgr2=0x7f2e400b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e47293640 1 --2- 172.21.15.79:0/956278987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 0x7f2e400b68d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2e4006d3b0 tx=0x7f2e3c02a1a0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e47293640 1 -- 172.21.15.79:0/956278987 shutdown_connections 2024-04-26T19:51:22.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e47293640 1 --2- 172.21.15.79:0/956278987 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 0x7f2e400b68d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e47293640 1 --2- 172.21.15.79:0/956278987 >> v2:172.21.15.83:3300/0 conn(0x7f2e400b64f0 0x7f2e40070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e47293640 1 -- 172.21.15.79:0/956278987 >> 172.21.15.79:0/956278987 conn(0x7f2e4006c770 msgr2=0x7f2e4006cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e47293640 1 -- 172.21.15.79:0/956278987 shutdown_connections 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.440+0000 7f2e47293640 1 -- 172.21.15.79:0/956278987 wait complete. 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e47293640 1 Processor -- start 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e47293640 1 -- start start 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e47293640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 0x7f2e4015a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e47293640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e400b64f0 0x7f2e400cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e47293640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e4015ffa0 0x7f2e400cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e47293640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4006f1e0 con 0x7f2e4006e800 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e47293640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4006ef00 con 0x7f2e4015ffa0 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e47293640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f2e4006f4c0 con 0x7f2e400b64f0 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e4502f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 0x7f2e4015a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e4502f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 0x7f2e4015a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57164/0 (socket says 172.21.15.79:57164) 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.442+0000 7f2e4502f640 1 -- 172.21.15.79:0/553204249 learned_addr learned my addr 172.21.15.79:0/553204249 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e4482e640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e400b64f0 0x7f2e400cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e45830640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e4015ffa0 0x7f2e400cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e4502f640 1 -- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e400b64f0 msgr2=0x7f2e400cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e4502f640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e400b64f0 0x7f2e400cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e4502f640 1 -- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e4015ffa0 msgr2=0x7f2e400cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e4502f640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e4015ffa0 0x7f2e400cc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e4502f640 1 -- 172.21.15.79:0/553204249 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e40143fe0 con 0x7f2e4006e800 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e4482e640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e400b64f0 0x7f2e400cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e45830640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e4015ffa0 0x7f2e400cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e4502f640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 0x7f2e4015a1d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000c7f0 tx=0x7f2e3000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e35ffb640 1 -- 172.21.15.79:0/553204249 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30015070 con 0x7f2e4006e800 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e35ffb640 1 -- 172.21.15.79:0/553204249 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e30007a30 con 0x7f2e4006e800 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e35ffb640 1 -- 172.21.15.79:0/553204249 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30007e00 con 0x7f2e4006e800 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4014e580 con 0x7f2e4006e800 2024-04-26T19:51:22.446 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.443+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e4014eb40 con 0x7f2e4006e800 2024-04-26T19:51:22.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.444+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e400506c0 con 0x7f2e4006e800 2024-04-26T19:51:22.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.444+0000 7f2e35ffb640 1 -- 172.21.15.79:0/553204249 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e30018070 con 0x7f2e4006e800 2024-04-26T19:51:22.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.445+0000 7f2e35ffb640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e2406c8f0 0x7f2e2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.445+0000 7f2e4482e640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e2406c8f0 0x7f2e2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.447 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.445+0000 7f2e35ffb640 1 -- 172.21.15.79:0/553204249 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2e3008bea0 con 0x7f2e4006e800 2024-04-26T19:51:22.448 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.445+0000 7f2e4482e640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e2406c8f0 0x7f2e2406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c034000 tx=0x7f2e3c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:22.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.449+0000 7f2e35ffb640 1 -- 172.21.15.79:0/553204249 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e30059f60 con 0x7f2e4006e800 2024-04-26T19:51:22.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.637+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2e40070780 con 0x7f2e4006e800 2024-04-26T19:51:22.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.637+0000 7f2e35ffb640 1 -- 172.21.15.79:0/553204249 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f2e30059860 con 0x7f2e4006e800 2024-04-26T19:51:22.638 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2024-04-26T19:51:22.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e2406c8f0 msgr2=0x7f2e2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e2406c8f0 0x7f2e2406edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c034000 tx=0x7f2e3c02e000 comp rx=0 tx=0).stop 2024-04-26T19:51:22.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 msgr2=0x7f2e4015a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 0x7f2e4015a1d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000c7f0 tx=0x7f2e3000cbf0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 shutdown_connections 2024-04-26T19:51:22.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e2406c8f0 0x7f2e2406edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e4015ffa0 0x7f2e400cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e400b64f0 0x7f2e400cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 --2- 172.21.15.79:0/553204249 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e4006e800 0x7f2e4015a1d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 >> 172.21.15.79:0/553204249 conn(0x7f2e4006c770 msgr2=0x7f2e4015ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:22.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 shutdown_connections 2024-04-26T19:51:22.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.639+0000 7f2e47293640 1 -- 172.21.15.79:0/553204249 wait complete. 2024-04-26T19:51:22.651 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T19:51:22.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:22.651 DEBUG:teuthology.orchestra.run.smithi079:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-26T19:51:22.675 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:51:22.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:22.675 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T19:51:22.731 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:51:22.731 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:22.750 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:51:22.750 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:22.809 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:51:22.809 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:22.868 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:22.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.964+0000 7ff3c51af640 1 Processor -- start 2024-04-26T19:51:22.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.964+0000 7ff3c51af640 1 -- start start 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.965+0000 7ff3c51af640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7ff3c0160770 0x7ff3c01637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.965+0000 7ff3c51af640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c0144ca0 con 0x7ff3c00b4220 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.965+0000 7ff3c51af640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3c0163e00 con 0x7ff3c0160390 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.965+0000 7ff3c51af640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7ff3c0144f30 con 0x7ff3c0160770 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.965+0000 7ff3be59c640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff3c00b4220 0x7ff3c01630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33306/0 (socket says 172.21.15.79:33306) 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.965+0000 7ff3be59c640 1 -- 172.21.15.79:0/2898065471 learned_addr learned my addr 172.21.15.79:0/2898065471 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.965+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2786154780 0 0) 0x7ff3c0144ca0 con 0x7ff3c00b4220 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.965+0000 7ff3bf59e640 1 --2- 172.21.15.79:0/2898065471 >> v2:172.21.15.83:3300/0 conn(0x7ff3c0160770 0x7ff3c01637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3a8003520 con 0x7ff3c00b4220 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 60997681 0 0) 0x7ff3a8003520 con 0x7ff3c00b4220 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 >> v2:172.21.15.83:3300/0 conn(0x7ff3c0160770 msgr2=0x7ff3c01637e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:22.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 --2- 172.21.15.79:0/2898065471 >> v2:172.21.15.83:3300/0 conn(0x7ff3c0160770 0x7ff3c01637e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 >> v1:172.21.15.83:6789/0 conn(0x7ff3c0160390 legacy=0x7ff3c00b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c0145210 con 0x7ff3c00b4220 2024-04-26T19:51:22.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7ff3b4001670 con 0x7ff3c00b4220 2024-04-26T19:51:22.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 >> v1:172.21.15.79:6789/0 conn(0x7ff3c00b4220 legacy=0x7ff3c01630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3a80039b0 0x7ff3a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a8004800 0x7ff3a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a80051c0 0x7ff3a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0144f30 con 0x7ff3a80039b0 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3a800c160 con 0x7ff3a80051c0 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7ff3a800c360 con 0x7ff3a8004800 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bed9d640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3a80039b0 0x7ff3a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3be59c640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a8004800 0x7ff3a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bf59e640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a80051c0 0x7ff3a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bed9d640 1 -- 172.21.15.79:0/2898065471 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a8004800 msgr2=0x7ff3a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bed9d640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a8004800 0x7ff3a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bed9d640 1 -- 172.21.15.79:0/2898065471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a80051c0 msgr2=0x7ff3a800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bed9d640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3a80051c0 0x7ff3a800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bed9d640 1 -- 172.21.15.79:0/2898065471 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3a8003520 con 0x7ff3a80039b0 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3bf59e640 1 -- 172.21.15.79:0/2898065471 reap_dead start 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.966+0000 7ff3be59c640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a8004800 0x7ff3a8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.967+0000 7ff3bed9d640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3a80039b0 0x7ff3a8003dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8001e50 tx=0x7ff3a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d530cbb9185df9a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.967+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8001a70 con 0x7ff3a80039b0 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.967+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b802ae40 con 0x7ff3a80039b0 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.967+0000 7ff3bdd9b640 1 -- 172.21.15.79:0/2898065471 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8003bb0 con 0x7ff3a80039b0 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.967+0000 7ff3c51af640 1 -- 172.21.15.79:0/2898065471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3a80039b0 msgr2=0x7ff3a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.967+0000 7ff3c51af640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3a80039b0 0x7ff3a8003dd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8001e50 tx=0x7ff3a8005830 comp rx=0 tx=0).stop 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.968+0000 7ff3c51af640 1 -- 172.21.15.79:0/2898065471 shutdown_connections 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.968+0000 7ff3c51af640 1 --2- 172.21.15.79:0/2898065471 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3a80039b0 0x7ff3a8003dd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.968+0000 7ff3c51af640 1 -- 172.21.15.79:0/2898065471 >> 172.21.15.79:0/2898065471 conn(0x7ff3c00b2720 msgr2=0x7ff3c0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:22.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.968+0000 7ff3c51af640 1 -- 172.21.15.79:0/2898065471 shutdown_connections 2024-04-26T19:51:22.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.968+0000 7ff3c51af640 1 -- 172.21.15.79:0/2898065471 wait complete. 2024-04-26T19:51:22.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.969+0000 7ff3c51af640 1 Processor -- start 2024-04-26T19:51:22.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.969+0000 7ff3c51af640 1 -- start start 2024-04-26T19:51:22.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3c51af640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a80039b0 0x7ff3c006f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3c51af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3c0148960 0x7ff3c00aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3c51af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c00af4d0 0x7ff3c015c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3c51af640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0145b90 con 0x7ff3c00af4d0 2024-04-26T19:51:22.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3c51af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0145a10 con 0x7ff3c0148960 2024-04-26T19:51:22.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3c51af640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7ff3c00b3b10 con 0x7ff3a80039b0 2024-04-26T19:51:22.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bf59e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c00af4d0 0x7ff3c015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bf59e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c00af4d0 0x7ff3c015c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57180/0 (socket says 172.21.15.79:57180) 2024-04-26T19:51:22.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bf59e640 1 -- 172.21.15.79:0/909725168 learned_addr learned my addr 172.21.15.79:0/909725168 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bed9d640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a80039b0 0x7ff3c006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3be59c640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3c0148960 0x7ff3c00aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bf59e640 1 -- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a80039b0 msgr2=0x7ff3c006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bf59e640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a80039b0 0x7ff3c006f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bf59e640 1 -- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3c0148960 msgr2=0x7ff3c00aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bf59e640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3c0148960 0x7ff3c00aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bf59e640 1 -- 172.21.15.79:0/909725168 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c015b5c0 con 0x7ff3c00af4d0 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bed9d640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a80039b0 0x7ff3c006f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3be59c640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3c0148960 0x7ff3c00aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:22.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3bf59e640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c00af4d0 0x7ff3c015c660 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00d33b0 tx=0x7ff3b0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.970+0000 7ff3a77fe640 1 -- 172.21.15.79:0/909725168 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0003610 con 0x7ff3c00af4d0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.971+0000 7ff3a77fe640 1 -- 172.21.15.79:0/909725168 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b001b070 con 0x7ff3c00af4d0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.971+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c015b8a0 con 0x7ff3c00af4d0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.971+0000 7ff3a77fe640 1 -- 172.21.15.79:0/909725168 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0003a10 con 0x7ff3c00af4d0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.971+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c015be60 con 0x7ff3c00af4d0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.971+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3c00506c0 con 0x7ff3c00af4d0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.972+0000 7ff3a77fe640 1 -- 172.21.15.79:0/909725168 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3b0003bb0 con 0x7ff3c00af4d0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.972+0000 7ff3a77fe640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff3ac06c8f0 0x7ff3ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.972+0000 7ff3a77fe640 1 -- 172.21.15.79:0/909725168 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff3b008c330 con 0x7ff3c00af4d0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.972+0000 7ff3bed9d640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff3ac06c8f0 0x7ff3ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:22.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.973+0000 7ff3bed9d640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff3ac06c8f0 0x7ff3ac06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8032000 tx=0x7ff3b802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:22.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:22.977+0000 7ff3a77fe640 1 -- 172.21.15.79:0/909725168 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff3b005a360 con 0x7ff3c00af4d0 2024-04-26T19:51:23.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.158+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff3c006d7b0 con 0x7ff3c00af4d0 2024-04-26T19:51:23.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.159+0000 7ff3a77fe640 1 -- 172.21.15.79:0/909725168 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7ff3b0059c60 con 0x7ff3c00af4d0 2024-04-26T19:51:23.160 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:23.160 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:23.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff3ac06c8f0 msgr2=0x7ff3ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff3ac06c8f0 0x7ff3ac06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8032000 tx=0x7ff3b802e000 comp rx=0 tx=0).stop 2024-04-26T19:51:23.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c00af4d0 msgr2=0x7ff3c015c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c00af4d0 0x7ff3c015c660 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00d33b0 tx=0x7ff3b0011920 comp rx=0 tx=0).stop 2024-04-26T19:51:23.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 shutdown_connections 2024-04-26T19:51:23.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff3ac06c8f0 0x7ff3ac06edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff3c00af4d0 0x7ff3c015c660 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff3c0148960 0x7ff3c00aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 --2- 172.21.15.79:0/909725168 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff3a80039b0 0x7ff3c006f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 >> 172.21.15.79:0/909725168 conn(0x7ff3c00b2720 msgr2=0x7ff3c0143b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:23.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 shutdown_connections 2024-04-26T19:51:23.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.161+0000 7ff3c51af640 1 -- 172.21.15.79:0/909725168 wait complete. 2024-04-26T19:51:23.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:23.174 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:23.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.273+0000 7f3ae12c8640 1 Processor -- start 2024-04-26T19:51:23.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.273+0000 7f3ae12c8640 1 -- start start 2024-04-26T19:51:23.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.273+0000 7f3ae12c8640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3adc0b0e60 0x7f3adc06f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.273+0000 7f3ae12c8640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b1240 0x7f3adc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.273+0000 7f3ae12c8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3adc15fc60 con 0x7f3adc0b0350 2024-04-26T19:51:23.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.273+0000 7f3ae12c8640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f3adc160870 con 0x7f3adc0b0e60 2024-04-26T19:51:23.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.273+0000 7f3ae12c8640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc15ff40 con 0x7f3adc0b1240 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.274+0000 7f3adaffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3adc0b0e60 0x7f3adc06f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.274+0000 7f3adaffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3adc0b0e60 0x7f3adc06f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57182/0 (socket says 172.21.15.79:57182) 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.274+0000 7f3adaffd640 1 -- 172.21.15.79:0/3156227246 learned_addr learned my addr 172.21.15.79:0/3156227246 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.274+0000 7f3adaffd640 1 -- 172.21.15.79:0/3156227246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b1240 msgr2=0x7f3adc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.274+0000 7f3ad3fff640 1 --2- 172.21.15.79:0/3156227246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b1240 0x7f3adc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.274+0000 7f3adaffd640 1 --2- 172.21.15.79:0/3156227246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b1240 0x7f3adc06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.274+0000 7f3adaffd640 1 -- 172.21.15.79:0/3156227246 >> v1:172.21.15.83:6789/0 conn(0x7f3adc0b0350 legacy=0x7f3adc0729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.274+0000 7f3adaffd640 1 -- 172.21.15.79:0/3156227246 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc160220 con 0x7f3adc0b0e60 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.274+0000 7f3adaffd640 1 --2- 172.21.15.79:0/3156227246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0e60 0x7f3adc06f0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009870 tx=0x7f3acc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e2bb324155b2c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ada7fc640 1 -- 172.21.15.79:0/3156227246 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc037070 con 0x7f3adc0b0e60 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ada7fc640 1 -- 172.21.15.79:0/3156227246 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3acc032440 con 0x7f3adc0b0e60 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ada7fc640 1 -- 172.21.15.79:0/3156227246 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc032810 con 0x7f3adc0b0e60 2024-04-26T19:51:23.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ae12c8640 1 -- 172.21.15.79:0/3156227246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0e60 msgr2=0x7f3adc06f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ae12c8640 1 --2- 172.21.15.79:0/3156227246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0e60 0x7f3adc06f0a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009870 tx=0x7f3acc02a180 comp rx=0 tx=0).stop 2024-04-26T19:51:23.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ae12c8640 1 -- 172.21.15.79:0/3156227246 shutdown_connections 2024-04-26T19:51:23.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ae12c8640 1 --2- 172.21.15.79:0/3156227246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b1240 0x7f3adc06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ae12c8640 1 --2- 172.21.15.79:0/3156227246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0e60 0x7f3adc06f0a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ae12c8640 1 -- 172.21.15.79:0/3156227246 >> 172.21.15.79:0/3156227246 conn(0x7f3adc06d5b0 msgr2=0x7f3adc0681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:23.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ae12c8640 1 -- 172.21.15.79:0/3156227246 shutdown_connections 2024-04-26T19:51:23.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.275+0000 7f3ae12c8640 1 -- 172.21.15.79:0/3156227246 wait complete. 2024-04-26T19:51:23.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.276+0000 7f3ae12c8640 1 Processor -- start 2024-04-26T19:51:23.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.276+0000 7f3ae12c8640 1 -- start start 2024-04-26T19:51:23.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3ae12c8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0350 0x7f3adc0c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3ae12c8640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b0e60 0x7f3adc0c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3ae12c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc0b1240 0x7f3adc0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3ae12c8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc073270 con 0x7f3adc0b0350 2024-04-26T19:51:23.281 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3ae12c8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3adc0730f0 con 0x7f3adc0b1240 2024-04-26T19:51:23.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3ae12c8640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3adc0733f0 con 0x7f3adc0b0e60 2024-04-26T19:51:23.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3adaffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0350 0x7f3adc0c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3adaffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0350 0x7f3adc0c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57196/0 (socket says 172.21.15.79:57196) 2024-04-26T19:51:23.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3adaffd640 1 -- 172.21.15.79:0/2388326321 learned_addr learned my addr 172.21.15.79:0/2388326321 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:23.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3ad3fff640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b0e60 0x7f3adc0c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3adb7fe640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc0b1240 0x7f3adc0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3adaffd640 1 -- 172.21.15.79:0/2388326321 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b0e60 msgr2=0x7f3adc0c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3adaffd640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b0e60 0x7f3adc0c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3adaffd640 1 -- 172.21.15.79:0/2388326321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc0b1240 msgr2=0x7f3adc0c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3adaffd640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc0b1240 0x7f3adc0c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.277+0000 7f3adaffd640 1 -- 172.21.15.79:0/2388326321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3adc15b330 con 0x7f3adc0b0350 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.278+0000 7f3adaffd640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0350 0x7f3adc0c62a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009870 tx=0x7f3acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.278+0000 7f3ad37fe640 1 -- 172.21.15.79:0/2388326321 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc041020 con 0x7f3adc0b0350 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.278+0000 7f3ad37fe640 1 -- 172.21.15.79:0/2388326321 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3acc030970 con 0x7f3adc0b0350 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.278+0000 7f3ad37fe640 1 -- 172.21.15.79:0/2388326321 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc037040 con 0x7f3adc0b0350 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.278+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3adc15b610 con 0x7f3adc0b0350 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.278+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3adc15bbd0 con 0x7f3adc0b0350 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.279+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3adc1639d0 con 0x7f3adc0b0350 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.279+0000 7f3ad37fe640 1 -- 172.21.15.79:0/2388326321 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3acc044020 con 0x7f3adc0b0350 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.280+0000 7f3ad37fe640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3abc06c8f0 0x7f3abc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.280+0000 7f3ad3fff640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3abc06c8f0 0x7f3abc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.280+0000 7f3ad3fff640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3abc06c8f0 0x7f3abc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4006fd0 tx=0x7f3ac4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:23.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.281+0000 7f3ad37fe640 1 -- 172.21.15.79:0/2388326321 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3acc0aab80 con 0x7f3adc0b0350 2024-04-26T19:51:23.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.285+0000 7f3ad37fe640 1 -- 172.21.15.79:0/2388326321 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3adc1639d0 con 0x7f3adc0b0350 2024-04-26T19:51:23.459 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.459+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3adc163f70 con 0x7f3adc0b0350 2024-04-26T19:51:23.460 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.459+0000 7f3ad37fe640 1 -- 172.21.15.79:0/2388326321 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f3adc163f70 con 0x7f3adc0b0350 2024-04-26T19:51:23.460 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3abc06c8f0 msgr2=0x7f3abc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3abc06c8f0 0x7f3abc06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4006fd0 tx=0x7f3ac4008040 comp rx=0 tx=0).stop 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0350 msgr2=0x7f3adc0c62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0350 0x7f3adc0c62a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009870 tx=0x7f3acc0046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 shutdown_connections 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3abc06c8f0 0x7f3abc06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3adc0b1240 0x7f3adc0c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3adc0b0e60 0x7f3adc0c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 --2- 172.21.15.79:0/2388326321 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3adc0b0350 0x7f3adc0c62a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 >> 172.21.15.79:0/2388326321 conn(0x7f3adc06d5b0 msgr2=0x7f3adc0728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 shutdown_connections 2024-04-26T19:51:23.462 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.461+0000 7f3ae12c8640 1 -- 172.21.15.79:0/2388326321 wait complete. 2024-04-26T19:51:23.475 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:23.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.574+0000 7fd53eadc640 1 Processor -- start 2024-04-26T19:51:23.575 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.574+0000 7fd53eadc640 1 -- start start 2024-04-26T19:51:23.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53eadc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd538163620 0x7fd538163a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53eadc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd53815f960 0x7fd53815fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53eadc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd538072220 0x7fd5380726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53eadc640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd5381603b0 con 0x7fd538072220 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53eadc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd538160690 con 0x7fd538163620 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53eadc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd5380aea70 con 0x7fd53815f960 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53d079640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd538072220 0x7fd5380726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53d079640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd538072220 0x7fd5380726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57198/0 (socket says 172.21.15.79:57198) 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53d079640 1 -- 172.21.15.79:0/3594094957 learned_addr learned my addr 172.21.15.79:0/3594094957 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd537fff640 1 --2- 172.21.15.79:0/3594094957 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd53815f960 0x7fd53815fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53c878640 1 --2- 172.21.15.79:0/3594094957 >> v2:172.21.15.83:3300/0 conn(0x7fd538163620 0x7fd538163a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53d079640 1 -- 172.21.15.79:0/3594094957 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd53815f960 msgr2=0x7fd53815fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53d079640 1 --2- 172.21.15.79:0/3594094957 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd53815f960 0x7fd53815fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53d079640 1 -- 172.21.15.79:0/3594094957 >> v2:172.21.15.83:3300/0 conn(0x7fd538163620 msgr2=0x7fd538163a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53d079640 1 --2- 172.21.15.79:0/3594094957 >> v2:172.21.15.83:3300/0 conn(0x7fd538163620 0x7fd538163a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.575+0000 7fd53d079640 1 -- 172.21.15.79:0/3594094957 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5380aed50 con 0x7fd538072220 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.576+0000 7fd53d079640 1 --2- 172.21.15.79:0/3594094957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 0x7fd5380726d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd538074370 tx=0x7fd53002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f0108cf7dd9e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.576+0000 7fd5377fe640 1 -- 172.21.15.79:0/3594094957 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd530036070 con 0x7fd538072220 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.576+0000 7fd5377fe640 1 -- 172.21.15.79:0/3594094957 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd530031600 con 0x7fd538072220 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.576+0000 7fd5377fe640 1 -- 172.21.15.79:0/3594094957 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5300319d0 con 0x7fd538072220 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.577+0000 7fd53eadc640 1 -- 172.21.15.79:0/3594094957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 msgr2=0x7fd5380726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.577+0000 7fd53eadc640 1 --2- 172.21.15.79:0/3594094957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 0x7fd5380726d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd538074370 tx=0x7fd53002a610 comp rx=0 tx=0).stop 2024-04-26T19:51:23.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.577+0000 7fd53eadc640 1 -- 172.21.15.79:0/3594094957 shutdown_connections 2024-04-26T19:51:23.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.577+0000 7fd53eadc640 1 --2- 172.21.15.79:0/3594094957 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 0x7fd5380726d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.577+0000 7fd53eadc640 1 --2- 172.21.15.79:0/3594094957 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd53815f960 0x7fd53815fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.577+0000 7fd53eadc640 1 --2- 172.21.15.79:0/3594094957 >> v2:172.21.15.83:3300/0 conn(0x7fd538163620 0x7fd538163a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.577+0000 7fd53eadc640 1 -- 172.21.15.79:0/3594094957 >> 172.21.15.79:0/3594094957 conn(0x7fd5380b2e30 msgr2=0x7fd5380b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:23.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.577+0000 7fd53eadc640 1 -- 172.21.15.79:0/3594094957 shutdown_connections 2024-04-26T19:51:23.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.577+0000 7fd53eadc640 1 -- 172.21.15.79:0/3594094957 wait complete. 2024-04-26T19:51:23.588 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.578+0000 7fd53eadc640 1 Processor -- start 2024-04-26T19:51:23.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.578+0000 7fd53eadc640 1 -- start start 2024-04-26T19:51:23.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53eadc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 0x7fd5381f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53eadc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd53815f960 0x7fd538162520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53eadc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd538163620 0x7fd538141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53eadc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd538146090 con 0x7fd538072220 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53eadc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd538145f10 con 0x7fd53815f960 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53eadc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd538146210 con 0x7fd538163620 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53c878640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 0x7fd5381f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53c878640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 0x7fd5381f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57212/0 (socket says 172.21.15.79:57212) 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53c878640 1 -- 172.21.15.79:0/229416059 learned_addr learned my addr 172.21.15.79:0/229416059 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53c878640 1 -- 172.21.15.79:0/229416059 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd538163620 msgr2=0x7fd538141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53c878640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd538163620 0x7fd538141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53c878640 1 -- 172.21.15.79:0/229416059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd53815f960 msgr2=0x7fd538162520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53c878640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd53815f960 0x7fd538162520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53c878640 1 -- 172.21.15.79:0/229416059 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5381616c0 con 0x7fd538072220 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd53c878640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 0x7fd5381f8c00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd52800e8a0 tx=0x7fd52800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:23.590 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.579+0000 7fd5357fa640 1 -- 172.21.15.79:0/229416059 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd528017070 con 0x7fd538072220 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.580+0000 7fd5357fa640 1 -- 172.21.15.79:0/229416059 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd52800c940 con 0x7fd538072220 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.580+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd538161960 con 0x7fd538072220 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.580+0000 7fd5357fa640 1 -- 172.21.15.79:0/229416059 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52800cd30 con 0x7fd538072220 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.580+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd538142850 con 0x7fd538072220 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.581+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5380cdfc0 con 0x7fd538072220 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.581+0000 7fd5357fa640 1 -- 172.21.15.79:0/229416059 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd52801a070 con 0x7fd538072220 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.581+0000 7fd5357fa640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd52406c8f0 0x7fd52406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.581+0000 7fd5357fa640 1 -- 172.21.15.79:0/229416059 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd52808bd50 con 0x7fd538072220 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.581+0000 7fd537fff640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd52406c8f0 0x7fd52406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.582+0000 7fd537fff640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd52406c8f0 0x7fd52406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd52c006a90 tx=0x7fd52c006a20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:23.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.586+0000 7fd5357fa640 1 -- 172.21.15.79:0/229416059 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd52808f030 con 0x7fd538072220 2024-04-26T19:51:23.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.760+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd5380506c0 con 0x7fd538072220 2024-04-26T19:51:23.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.761+0000 7fd5357fa640 1 -- 172.21.15.79:0/229416059 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fd528059e10 con 0x7fd538072220 2024-04-26T19:51:23.762 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:23.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd52406c8f0 msgr2=0x7fd52406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd52406c8f0 0x7fd52406edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd52c006a90 tx=0x7fd52c006a20 comp rx=0 tx=0).stop 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 msgr2=0x7fd5381f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 0x7fd5381f8c00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd52800e8a0 tx=0x7fd52800ed60 comp rx=0 tx=0).stop 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 shutdown_connections 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd52406c8f0 0x7fd52406edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd538163620 0x7fd538141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd53815f960 0x7fd538162520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 --2- 172.21.15.79:0/229416059 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd538072220 0x7fd5381f8c00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 >> 172.21.15.79:0/229416059 conn(0x7fd5380b2e30 msgr2=0x7fd5380b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 shutdown_connections 2024-04-26T19:51:23.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.763+0000 7fd53eadc640 1 -- 172.21.15.79:0/229416059 wait complete. 2024-04-26T19:51:23.775 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:23.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.872+0000 7f223f340640 1 Processor -- start 2024-04-26T19:51:23.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.872+0000 7f223f340640 1 -- start start 2024-04-26T19:51:23.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.873+0000 7f223f340640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380afd10 0x7f223806ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.873+0000 7f223f340640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22380cfdd0 con 0x7f2238141d10 2024-04-26T19:51:23.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.873+0000 7f223f340640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22380d00b0 con 0x7f22380af8b0 2024-04-26T19:51:23.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.873+0000 7f223f340640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f22380d0390 con 0x7f22380afd10 2024-04-26T19:51:23.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.873+0000 7f223d0dc640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f2238141d10 0x7f22380ce2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33318/0 (socket says 172.21.15.79:33318) 2024-04-26T19:51:23.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.873+0000 7f223d0dc640 1 -- 172.21.15.79:0/2497638235 learned_addr learned my addr 172.21.15.79:0/2497638235 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:23.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.873+0000 7f223d8dd640 1 --2- 172.21.15.79:0/2497638235 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380afd10 0x7f223806ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.873+0000 7f222ffff640 1 -- 172.21.15.79:0/2497638235 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3769836283 0 0) 0x7f22380cfdd0 con 0x7f2238141d10 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.873+0000 7f222ffff640 1 -- 172.21.15.79:0/2497638235 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2224003520 con 0x7f2238141d10 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.874+0000 7f222ffff640 1 -- 172.21.15.79:0/2497638235 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2889551640 0 0) 0x7f22380d00b0 con 0x7f22380af8b0 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.874+0000 7f223d8dd640 1 -- 172.21.15.79:0/2497638235 >> v1:172.21.15.83:6789/0 conn(0x7f22380af8b0 legacy=0x7f223806e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.874+0000 7f223d8dd640 1 -- 172.21.15.79:0/2497638235 >> v1:172.21.15.79:6789/0 conn(0x7f2238141d10 legacy=0x7f22380ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.874+0000 7f223d8dd640 1 -- 172.21.15.79:0/2497638235 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22380d0670 con 0x7f22380afd10 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.874+0000 7f223d8dd640 1 --2- 172.21.15.79:0/2497638235 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380afd10 0x7f223806ef90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22300097d0 tx=0x7f223002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5176c82bd4c5bae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.874+0000 7f222ffff640 1 -- 172.21.15.79:0/2497638235 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230037070 con 0x7f22380afd10 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.875+0000 7f222ffff640 1 -- 172.21.15.79:0/2497638235 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2230032440 con 0x7f22380afd10 2024-04-26T19:51:23.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.875+0000 7f222ffff640 1 -- 172.21.15.79:0/2497638235 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22300327f0 con 0x7f22380afd10 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.875+0000 7f223f340640 1 -- 172.21.15.79:0/2497638235 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380afd10 msgr2=0x7f223806ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.875+0000 7f223f340640 1 --2- 172.21.15.79:0/2497638235 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380afd10 0x7f223806ef90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22300097d0 tx=0x7f223002a210 comp rx=0 tx=0).stop 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.875+0000 7f223f340640 1 -- 172.21.15.79:0/2497638235 shutdown_connections 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.875+0000 7f223f340640 1 --2- 172.21.15.79:0/2497638235 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380afd10 0x7f223806ef90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.875+0000 7f223f340640 1 -- 172.21.15.79:0/2497638235 >> 172.21.15.79:0/2497638235 conn(0x7f22380b2540 msgr2=0x7f22380b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.875+0000 7f223f340640 1 -- 172.21.15.79:0/2497638235 shutdown_connections 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.875+0000 7f223f340640 1 -- 172.21.15.79:0/2497638235 wait complete. 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.876+0000 7f223f340640 1 Processor -- start 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.876+0000 7f223f340640 1 -- start start 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223f340640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380af8b0 0x7f2238139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223f340640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22380afd10 0x7f223806df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223f340640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2238141d10 0x7f22380b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223f340640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f22380b01a0 con 0x7f22380afd10 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223f340640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f22380d0c30 con 0x7f2238141d10 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223f340640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f22380cddc0 con 0x7f22380af8b0 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223c8db640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22380afd10 0x7f223806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223c8db640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22380afd10 0x7f223806df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57214/0 (socket says 172.21.15.79:57214) 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223c8db640 1 -- 172.21.15.79:0/404336665 learned_addr learned my addr 172.21.15.79:0/404336665 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223d0dc640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380af8b0 0x7f2238139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223d8dd640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2238141d10 0x7f22380b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223c8db640 1 -- 172.21.15.79:0/404336665 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380af8b0 msgr2=0x7f2238139f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223c8db640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380af8b0 0x7f2238139f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223c8db640 1 -- 172.21.15.79:0/404336665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2238141d10 msgr2=0x7f22380b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223c8db640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2238141d10 0x7f22380b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.877+0000 7f223c8db640 1 -- 172.21.15.79:0/404336665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22381555a0 con 0x7f22380afd10 2024-04-26T19:51:23.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.878+0000 7f223c8db640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22380afd10 0x7f223806df80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f222800c4b0 tx=0x7f222800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:23.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.878+0000 7f222dffb640 1 -- 172.21.15.79:0/404336665 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228017070 con 0x7f22380afd10 2024-04-26T19:51:23.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.878+0000 7f222dffb640 1 -- 172.21.15.79:0/404336665 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2228005480 con 0x7f22380afd10 2024-04-26T19:51:23.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.878+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223806e4c0 con 0x7f22380afd10 2024-04-26T19:51:23.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.878+0000 7f222dffb640 1 -- 172.21.15.79:0/404336665 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228003830 con 0x7f22380afd10 2024-04-26T19:51:23.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.878+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f223815e870 con 0x7f22380afd10 2024-04-26T19:51:23.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.879+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22380506c0 con 0x7f22380afd10 2024-04-26T19:51:23.885 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.879+0000 7f222dffb640 1 -- 172.21.15.79:0/404336665 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f222801a070 con 0x7f22380afd10 2024-04-26T19:51:23.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.880+0000 7f222dffb640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f221c06c8f0 0x7f221c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:23.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.880+0000 7f222dffb640 1 -- 172.21.15.79:0/404336665 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f222808b230 con 0x7f22380afd10 2024-04-26T19:51:23.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.880+0000 7f223d0dc640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f221c06c8f0 0x7f221c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:23.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.880+0000 7f223d0dc640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f221c06c8f0 0x7f221c06edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2234005c00 tx=0x7f2234009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:23.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:23.884+0000 7f222dffb640 1 -- 172.21.15.79:0/404336665 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2228058ca0 con 0x7f22380afd10 2024-04-26T19:51:24.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.056+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2238072180 con 0x7f22380afd10 2024-04-26T19:51:24.057 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.057+0000 7f222dffb640 1 -- 172.21.15.79:0/404336665 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f222805cfa0 con 0x7f22380afd10 2024-04-26T19:51:24.057 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:24.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f221c06c8f0 msgr2=0x7f221c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f221c06c8f0 0x7f221c06edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2234005c00 tx=0x7f2234009040 comp rx=0 tx=0).stop 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22380afd10 msgr2=0x7f223806df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22380afd10 0x7f223806df80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f222800c4b0 tx=0x7f222800c970 comp rx=0 tx=0).stop 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 shutdown_connections 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f221c06c8f0 0x7f221c06edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2238141d10 0x7f22380b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f22380afd10 0x7f223806df80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 --2- 172.21.15.79:0/404336665 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f22380af8b0 0x7f2238139f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 >> 172.21.15.79:0/404336665 conn(0x7f22380b2540 msgr2=0x7f22380723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 shutdown_connections 2024-04-26T19:51:24.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.059+0000 7f223f340640 1 -- 172.21.15.79:0/404336665 wait complete. 2024-04-26T19:51:24.070 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:24.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.169+0000 7f2644486640 1 Processor -- start 2024-04-26T19:51:24.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.169+0000 7f2644486640 1 -- start start 2024-04-26T19:51:24.173 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.170+0000 7f2644486640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f263c160610 0x7f263c0b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.170+0000 7f2644486640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c06f210 0x7f263c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.170+0000 7f2644486640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f263c0ba6f0 con 0x7f263c1598c0 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.170+0000 7f2644486640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f263c06fbc0 con 0x7f263c160610 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.170+0000 7f2644486640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f263c0d5100 con 0x7f263c06f210 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.170+0000 7f2642222640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f263c1598c0 0x7f263c0b9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33328/0 (socket says 172.21.15.79:33328) 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.170+0000 7f2642222640 1 -- 172.21.15.79:0/3425718373 learned_addr learned my addr 172.21.15.79:0/3425718373 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279522076 0 0) 0x7f263c0ba6f0 con 0x7f263c1598c0 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2628003520 con 0x7f263c1598c0 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2642a23640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c06f210 0x7f263c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641a21640 1 --2- 172.21.15.79:0/3425718373 >> v2:172.21.15.83:3300/0 conn(0x7f263c160610 0x7f263c0b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1288525272 0 0) 0x7f2628003520 con 0x7f263c1598c0 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c06f210 msgr2=0x7f263c06f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c06f210 0x7f263c06f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 >> v2:172.21.15.83:3300/0 conn(0x7f263c160610 msgr2=0x7f263c0b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 --2- 172.21.15.79:0/3425718373 >> v2:172.21.15.83:3300/0 conn(0x7f263c160610 0x7f263c0b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c0ba410 con 0x7f263c1598c0 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f26340019f0 con 0x7f263c1598c0 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 >> v1:172.21.15.79:6789/0 conn(0x7f263c1598c0 legacy=0x7f263c0b9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c06f210 0x7f26280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2628003f30 0x7f2628004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628004d40 0x7f26280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f263c06fbc0 con 0x7f2628003f30 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f263c0d5100 con 0x7f2628004d40 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f262800ba10 con 0x7f263c06f210 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641a21640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2628003f30 0x7f2628004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2642222640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c06f210 0x7f26280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2642a23640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628004d40 0x7f26280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641a21640 1 -- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c06f210 msgr2=0x7f26280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:24.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641a21640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c06f210 0x7f26280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641a21640 1 -- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628004d40 msgr2=0x7f26280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641a21640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628004d40 0x7f26280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2641a21640 1 -- 172.21.15.79:0/3425718373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2628003520 con 0x7f2628003f30 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.171+0000 7f2642a23640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628004d40 0x7f26280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.172+0000 7f2641a21640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2628003f30 0x7f2628004800 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f262c009af0 tx=0x7f2628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7a8317c083041b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.172+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c036070 con 0x7f2628003f30 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.172+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c031400 con 0x7f2628003f30 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.172+0000 7f2641220640 1 -- 172.21.15.79:0/3425718373 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c009690 con 0x7f2628003f30 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.173+0000 7f2644486640 1 -- 172.21.15.79:0/3425718373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2628003f30 msgr2=0x7f2628004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.173+0000 7f2644486640 1 --2- 172.21.15.79:0/3425718373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2628003f30 0x7f2628004800 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f262c009af0 tx=0x7f2628005830 comp rx=0 tx=0).stop 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.173+0000 7f2642a23640 1 -- 172.21.15.79:0/3425718373 reap_dead start 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.173+0000 7f2644486640 1 -- 172.21.15.79:0/3425718373 shutdown_connections 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.173+0000 7f2644486640 1 -- 172.21.15.79:0/3425718373 >> 172.21.15.79:0/3425718373 conn(0x7f263c0b4930 msgr2=0x7f263c0b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.173+0000 7f2644486640 1 -- 172.21.15.79:0/3425718373 shutdown_connections 2024-04-26T19:51:24.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.173+0000 7f2644486640 1 -- 172.21.15.79:0/3425718373 wait complete. 2024-04-26T19:51:24.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.174+0000 7f2644486640 1 Processor -- start 2024-04-26T19:51:24.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.174+0000 7f2644486640 1 -- start start 2024-04-26T19:51:24.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2644486640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628003f30 0x7f263c15cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2644486640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f263c0b7dc0 0x7f263c15e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2644486640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c15a120 0x7f263c144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2644486640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f263c159ca0 con 0x7f263c0b7dc0 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2644486640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f263c0b8940 con 0x7f2628003f30 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2644486640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f263c159e20 con 0x7f263c15a120 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2641a21640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f263c0b7dc0 0x7f263c15e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2641a21640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f263c0b7dc0 0x7f263c15e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57230/0 (socket says 172.21.15.79:57230) 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2641a21640 1 -- 172.21.15.79:0/2008471539 learned_addr learned my addr 172.21.15.79:0/2008471539 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2642222640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628003f30 0x7f263c15cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2642a23640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c15a120 0x7f263c144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2641a21640 1 -- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c15a120 msgr2=0x7f263c144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2641a21640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c15a120 0x7f263c144830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2641a21640 1 -- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628003f30 msgr2=0x7f263c15cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2641a21640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628003f30 0x7f263c15cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2641a21640 1 -- 172.21.15.79:0/2008471539 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263c15c1e0 con 0x7f263c0b7dc0 2024-04-26T19:51:24.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2642222640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628003f30 0x7f263c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2642a23640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c15a120 0x7f263c144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2641a21640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f263c0b7dc0 0x7f263c15e980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f262c034040 tx=0x7f262c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2632ffd640 1 -- 172.21.15.79:0/2008471539 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c040020 con 0x7f263c0b7dc0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2632ffd640 1 -- 172.21.15.79:0/2008471539 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262c003810 con 0x7f263c0b7dc0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263c15c4c0 con 0x7f263c0b7dc0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2632ffd640 1 -- 172.21.15.79:0/2008471539 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c036040 con 0x7f263c0b7dc0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.175+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263c15ca80 con 0x7f263c0b7dc0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.176+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f263c0506c0 con 0x7f263c0b7dc0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.177+0000 7f2632ffd640 1 -- 172.21.15.79:0/2008471539 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f262c043020 con 0x7f263c0b7dc0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.177+0000 7f2632ffd640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f262006c8f0 0x7f262006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.177+0000 7f2632ffd640 1 -- 172.21.15.79:0/2008471539 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f262c0abe30 con 0x7f263c0b7dc0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.177+0000 7f2642222640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f262006c8f0 0x7f262006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.178+0000 7f2642222640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f262006c8f0 0x7f262006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2634002230 tx=0x7f2634006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:24.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.182+0000 7f2632ffd640 1 -- 172.21.15.79:0/2008471539 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f262c04a070 con 0x7f263c0b7dc0 2024-04-26T19:51:24.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.358+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f263c15be60 con 0x7f263c0b7dc0 2024-04-26T19:51:24.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.358+0000 7f2632ffd640 1 -- 172.21.15.79:0/2008471539 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f262c079ef0 con 0x7f263c0b7dc0 2024-04-26T19:51:24.359 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:24.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f262006c8f0 msgr2=0x7f262006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f262006c8f0 0x7f262006edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2634002230 tx=0x7f2634006040 comp rx=0 tx=0).stop 2024-04-26T19:51:24.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f263c0b7dc0 msgr2=0x7f263c15e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f263c0b7dc0 0x7f263c15e980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f262c034040 tx=0x7f262c004730 comp rx=0 tx=0).stop 2024-04-26T19:51:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 shutdown_connections 2024-04-26T19:51:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f262006c8f0 0x7f262006edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f263c15a120 0x7f263c144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f263c0b7dc0 0x7f263c15e980 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 --2- 172.21.15.79:0/2008471539 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2628003f30 0x7f263c15cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 >> 172.21.15.79:0/2008471539 conn(0x7f263c0b4930 msgr2=0x7f263c0b9720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 shutdown_connections 2024-04-26T19:51:24.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.360+0000 7f2644486640 1 -- 172.21.15.79:0/2008471539 wait complete. 2024-04-26T19:51:24.372 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T19:51:24.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.468+0000 7fc91361d640 1 Processor -- start 2024-04-26T19:51:24.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.468+0000 7fc91361d640 1 -- start start 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.469+0000 7fc91361d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc90c0b4250 0x7fc90c167ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.469+0000 7fc91361d640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c0aa880 con 0x7fc90c1402c0 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.469+0000 7fc91361d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc90c0aab60 con 0x7fc90c13fee0 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.469+0000 7fc91361d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc90c0aae40 con 0x7fc90c0b4250 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.469+0000 7fc910bb8640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fc90c1402c0 0x7fc90c0b3b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33340/0 (socket says 172.21.15.79:33340) 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.469+0000 7fc910bb8640 1 -- 172.21.15.79:0/669391756 learned_addr learned my addr 172.21.15.79:0/669391756 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 71563881 0 0) 0x7fc90c0aa880 con 0x7fc90c1402c0 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc911bba640 1 --2- 172.21.15.79:0/669391756 >> v2:172.21.15.83:3300/0 conn(0x7fc90c0b4250 0x7fc90c167ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc8f4003520 con 0x7fc90c1402c0 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3930084 0 0) 0x7fc8f4003520 con 0x7fc90c1402c0 2024-04-26T19:51:24.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 >> v2:172.21.15.83:3300/0 conn(0x7fc90c0b4250 msgr2=0x7fc90c167ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 --2- 172.21.15.79:0/669391756 >> v2:172.21.15.83:3300/0 conn(0x7fc90c0b4250 0x7fc90c167ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 >> v1:172.21.15.83:6789/0 conn(0x7fc90c13fee0 legacy=0x7fc90c1432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0ab120 con 0x7fc90c1402c0 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fc900001670 con 0x7fc90c1402c0 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 >> v1:172.21.15.79:6789/0 conn(0x7fc90c1402c0 legacy=0x7fc90c0b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40039b0 0x7fc8f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc8f4004800 0x7fc8f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f40051c0 0x7fc8f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c0aae40 con 0x7fc8f40051c0 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f400c110 con 0x7fc8f40039b0 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f400c310 con 0x7fc8f4004800 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc911bba640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f40051c0 0x7fc8f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc9113b9640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40039b0 0x7fc8f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc910bb8640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc8f4004800 0x7fc8f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc911bba640 1 -- 172.21.15.79:0/669391756 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc8f4004800 msgr2=0x7fc8f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc911bba640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc8f4004800 0x7fc8f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc911bba640 1 -- 172.21.15.79:0/669391756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40039b0 msgr2=0x7fc8f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc911bba640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40039b0 0x7fc8f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc911bba640 1 -- 172.21.15.79:0/669391756 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f4003520 con 0x7fc8f40051c0 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.470+0000 7fc910bb8640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc8f4004800 0x7fc8f4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.471+0000 7fc911bba640 1 -- 172.21.15.79:0/669391756 reap_dead start 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.471+0000 7fc911bba640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f40051c0 0x7fc8f400b9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc90400ba10 tx=0x7fc8f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37caa5a8d394924 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.471+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904030070 con 0x7fc8f40051c0 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.471+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc90402ae00 con 0x7fc8f40051c0 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.471+0000 7fc90bfff640 1 -- 172.21.15.79:0/669391756 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90400b610 con 0x7fc8f40051c0 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.472+0000 7fc91361d640 1 -- 172.21.15.79:0/669391756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f40051c0 msgr2=0x7fc8f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.472+0000 7fc91361d640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f40051c0 0x7fc8f400b9c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc90400ba10 tx=0x7fc8f4005830 comp rx=0 tx=0).stop 2024-04-26T19:51:24.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.472+0000 7fc91361d640 1 -- 172.21.15.79:0/669391756 shutdown_connections 2024-04-26T19:51:24.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.472+0000 7fc91361d640 1 --2- 172.21.15.79:0/669391756 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc8f40051c0 0x7fc8f400b9c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.472+0000 7fc91361d640 1 -- 172.21.15.79:0/669391756 >> 172.21.15.79:0/669391756 conn(0x7fc90c0b2720 msgr2=0x7fc90c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:24.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.472+0000 7fc91361d640 1 -- 172.21.15.79:0/669391756 shutdown_connections 2024-04-26T19:51:24.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.472+0000 7fc91361d640 1 -- 172.21.15.79:0/669391756 wait complete. 2024-04-26T19:51:24.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.473+0000 7fc91361d640 1 Processor -- start 2024-04-26T19:51:24.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.473+0000 7fc91361d640 1 -- start start 2024-04-26T19:51:24.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc91361d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40051c0 0x7fc90c153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc91361d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc90c1547c0 0x7fc90c13dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc91361d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc90c1f6020 0x7fc90c1f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc91361d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c0ab920 con 0x7fc90c1547c0 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc91361d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c0d1500 con 0x7fc8f40051c0 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc91361d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc90c0aa4c0 con 0x7fc90c1f6020 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc910bb8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc90c1547c0 0x7fc90c13dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc910bb8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc90c1547c0 0x7fc90c13dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57244/0 (socket says 172.21.15.79:57244) 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc910bb8640 1 -- 172.21.15.79:0/3579063183 learned_addr learned my addr 172.21.15.79:0/3579063183 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc911bba640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc90c1f6020 0x7fc90c1f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc9113b9640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40051c0 0x7fc90c153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc910bb8640 1 -- 172.21.15.79:0/3579063183 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc90c1f6020 msgr2=0x7fc90c1f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc910bb8640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc90c1f6020 0x7fc90c1f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc910bb8640 1 -- 172.21.15.79:0/3579063183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40051c0 msgr2=0x7fc90c153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc910bb8640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40051c0 0x7fc90c153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc910bb8640 1 -- 172.21.15.79:0/3579063183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c0b1920 con 0x7fc90c1547c0 2024-04-26T19:51:24.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc910bb8640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc90c1547c0 0x7fc90c13dc20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e730 tx=0x7fc90000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc909ffb640 1 -- 172.21.15.79:0/3579063183 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90000ee40 con 0x7fc90c1547c0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc909ffb640 1 -- 172.21.15.79:0/3579063183 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc900018070 con 0x7fc90c1547c0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c153210 con 0x7fc90c1547c0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.474+0000 7fc909ffb640 1 -- 172.21.15.79:0/3579063183 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc90000ca70 con 0x7fc90c1547c0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.475+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c1537d0 con 0x7fc90c1547c0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.476+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c1539d0 con 0x7fc90c1547c0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.476+0000 7fc909ffb640 1 -- 172.21.15.79:0/3579063183 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc90000cc10 con 0x7fc90c1547c0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.476+0000 7fc909ffb640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc8f806c920 0x7fc8f806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.476+0000 7fc909ffb640 1 -- 172.21.15.79:0/3579063183 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc90008b9e0 con 0x7fc90c1547c0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.476+0000 7fc9113b9640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc8f806c920 0x7fc8f806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:24.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.477+0000 7fc9113b9640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc8f806c920 0x7fc8f806ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc003e50 tx=0x7fc8fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:24.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.481+0000 7fc909ffb640 1 -- 172.21.15.79:0/3579063183 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc90c1539d0 con 0x7fc90c1547c0 2024-04-26T19:51:24.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.665+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc90c0506c0 con 0x7fc90c1547c0 2024-04-26T19:51:24.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.666+0000 7fc909ffb640 1 -- 172.21.15.79:0/3579063183 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fc900059a10 con 0x7fc90c1547c0 2024-04-26T19:51:24.666 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2024-04-26T19:51:24.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.667+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc8f806c920 msgr2=0x7fc8f806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.667+0000 7fc91361d640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc8f806c920 0x7fc8f806ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc003e50 tx=0x7fc8fc009040 comp rx=0 tx=0).stop 2024-04-26T19:51:24.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.667+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc90c1547c0 msgr2=0x7fc90c13dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:24.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.667+0000 7fc91361d640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc90c1547c0 0x7fc90c13dc20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e730 tx=0x7fc90000ebf0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.668+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 shutdown_connections 2024-04-26T19:51:24.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.668+0000 7fc91361d640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc8f806c920 0x7fc8f806ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.668+0000 7fc91361d640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc90c1f6020 0x7fc90c1f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.668+0000 7fc91361d640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc90c1547c0 0x7fc90c13dc20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.668+0000 7fc91361d640 1 --2- 172.21.15.79:0/3579063183 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc8f40051c0 0x7fc90c153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:24.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.668+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 >> 172.21.15.79:0/3579063183 conn(0x7fc90c0b2720 msgr2=0x7fc90c142770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:24.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.668+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 shutdown_connections 2024-04-26T19:51:24.669 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:24.668+0000 7fc91361d640 1 -- 172.21.15.79:0/3579063183 wait complete. 2024-04-26T19:51:24.679 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T19:51:24.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:24.679 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-26T19:51:24.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:51:24.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:24.711 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T19:51:24.782 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:51:24.782 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:24.797 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:51:24.798 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:24.852 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:51:24.852 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:24.912 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:25.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.048+0000 7f43697f0640 1 Processor -- start 2024-04-26T19:51:25.049 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.048+0000 7f43697f0640 1 -- start start 2024-04-26T19:51:25.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.049+0000 7f43697f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f436415f7f0 0x7f43640c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.049+0000 7f43697f0640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f436415fbd0 con 0x7f4364070340 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.049+0000 7f43697f0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4364070840 con 0x7f43640d1530 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.049+0000 7f43697f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4364070b20 con 0x7f436415f7f0 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.049+0000 7f43627fc640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4364070340 0x7f43640728c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33346/0 (socket says 172.21.15.79:33346) 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.049+0000 7f43627fc640 1 -- 172.21.15.79:0/966019163 learned_addr learned my addr 172.21.15.79:0/966019163 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4362ffd640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f436415f7f0 0x7f43640c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1246719994 0 0) 0x7f436415fbd0 con 0x7f4364070340 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f434c003520 con 0x7f4364070340 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3613096695 0 0) 0x7f434c003520 con 0x7f4364070340 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f436415f7f0 msgr2=0x7f43640c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f436415f7f0 0x7f43640c6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 >> v1:172.21.15.83:6789/0 conn(0x7f43640d1530 legacy=0x7f43640cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4364070e00 con 0x7f4364070340 2024-04-26T19:51:25.053 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f4358001670 con 0x7f4364070340 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 >> v1:172.21.15.79:6789/0 conn(0x7f4364070340 legacy=0x7f43640728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f436415f7f0 0x7f434c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434c003f20 0x7f434c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f434c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4364070b20 con 0x7f434c004d40 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f434c00ba20 con 0x7f434c003f20 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f434c00bc20 con 0x7f436415f7f0 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f43637fe640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f434c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f4362ffd640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f436415f7f0 0x7f434c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f43627fc640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434c003f20 0x7f434c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f43637fe640 1 -- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f436415f7f0 msgr2=0x7f434c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f43637fe640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f436415f7f0 0x7f434c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f43637fe640 1 -- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434c003f20 msgr2=0x7f434c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:25.054 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f43637fe640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f434c003f20 0x7f434c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.050+0000 7f43637fe640 1 -- 172.21.15.79:0/966019163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434c003520 con 0x7f434c004d40 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.051+0000 7f43637fe640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f434c0051c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f435c001860 tx=0x7f434c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a373e6d98fc435 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.051+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c037070 con 0x7f434c004d40 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.051+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f435c02abf0 con 0x7f434c004d40 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.051+0000 7f4361ffb640 1 -- 172.21.15.79:0/966019163 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435c001430 con 0x7f434c004d40 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.052+0000 7f43697f0640 1 -- 172.21.15.79:0/966019163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 msgr2=0x7f434c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.052+0000 7f43697f0640 1 --2- 172.21.15.79:0/966019163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f434c0051c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f435c001860 tx=0x7f434c005830 comp rx=0 tx=0).stop 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.052+0000 7f43637fe640 1 -- 172.21.15.79:0/966019163 reap_dead start 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.052+0000 7f43697f0640 1 -- 172.21.15.79:0/966019163 shutdown_connections 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.052+0000 7f43697f0640 1 -- 172.21.15.79:0/966019163 >> 172.21.15.79:0/966019163 conn(0x7f436406d290 msgr2=0x7f43640d0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:25.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.052+0000 7f43697f0640 1 -- 172.21.15.79:0/966019163 shutdown_connections 2024-04-26T19:51:25.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.052+0000 7f43697f0640 1 -- 172.21.15.79:0/966019163 wait complete. 2024-04-26T19:51:25.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.053+0000 7f43697f0640 1 Processor -- start 2024-04-26T19:51:25.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.053+0000 7f43697f0640 1 -- start start 2024-04-26T19:51:25.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f43697f0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f43641564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.058 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f43697f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4364156a20 0x7f43641f55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f43697f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f43641f5b00 0x7f4364156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f43697f0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f436406d930 con 0x7f434c004d40 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f43697f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43640723a0 con 0x7f4364156a20 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f43697f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f436406dab0 con 0x7f43641f5b00 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f4362ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f43641564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f4362ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f43641564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57272/0 (socket says 172.21.15.79:57272) 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f4362ffd640 1 -- 172.21.15.79:0/2849102661 learned_addr learned my addr 172.21.15.79:0/2849102661 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f43637fe640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f43641f5b00 0x7f4364156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f43627fc640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4364156a20 0x7f43641f55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f4362ffd640 1 -- 172.21.15.79:0/2849102661 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f43641f5b00 msgr2=0x7f4364156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f4362ffd640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f43641f5b00 0x7f4364156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f4362ffd640 1 -- 172.21.15.79:0/2849102661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4364156a20 msgr2=0x7f43641f55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f4362ffd640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4364156a20 0x7f43641f55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.059 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f4362ffd640 1 -- 172.21.15.79:0/2849102661 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43641591f0 con 0x7f434c004d40 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f43627fc640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4364156a20 0x7f43641f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.054+0000 7f4362ffd640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f43641564e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4354011e60 tx=0x7f435400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.055+0000 7f434b7fe640 1 -- 172.21.15.79:0/2849102661 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354015630 con 0x7f434c004d40 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.055+0000 7f434b7fe640 1 -- 172.21.15.79:0/2849102661 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43540157d0 con 0x7f434c004d40 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.055+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43641594d0 con 0x7f434c004d40 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.055+0000 7f434b7fe640 1 -- 172.21.15.79:0/2849102661 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354015ba0 con 0x7f434c004d40 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.055+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f436415a1e0 con 0x7f434c004d40 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.056+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43640506c0 con 0x7f434c004d40 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.056+0000 7f434b7fe640 1 -- 172.21.15.79:0/2849102661 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f435401d400 con 0x7f434c004d40 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.056+0000 7f434b7fe640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f435006c920 0x7f435006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.056+0000 7f434b7fe640 1 -- 172.21.15.79:0/2849102661 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f435408d160 con 0x7f434c004d40 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.057+0000 7f43627fc640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f435006c920 0x7f435006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.060 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.057+0000 7f43627fc640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f435006c920 0x7f435006ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4364156e40 tx=0x7f4358009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.061+0000 7f434b7fe640 1 -- 172.21.15.79:0/2849102661 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4354019030 con 0x7f434c004d40 2024-04-26T19:51:25.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.235+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f436415eb40 con 0x7f434c004d40 2024-04-26T19:51:25.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.236+0000 7f434b7fe640 1 -- 172.21.15.79:0/2849102661 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7f4354010040 con 0x7f434c004d40 2024-04-26T19:51:25.237 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:25.237 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:25.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f435006c920 msgr2=0x7f435006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f435006c920 0x7f435006ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4364156e40 tx=0x7f4358009040 comp rx=0 tx=0).stop 2024-04-26T19:51:25.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 msgr2=0x7f43641564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f43641564e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4354011e60 tx=0x7f435400f580 comp rx=0 tx=0).stop 2024-04-26T19:51:25.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 shutdown_connections 2024-04-26T19:51:25.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f435006c920 0x7f435006ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f43641f5b00 0x7f4364156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4364156a20 0x7f43641f55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 --2- 172.21.15.79:0/2849102661 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f434c004d40 0x7f43641564e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.240 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 >> 172.21.15.79:0/2849102661 conn(0x7f436406d290 msgr2=0x7f436415de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:25.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 shutdown_connections 2024-04-26T19:51:25.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.238+0000 7f43697f0640 1 -- 172.21.15.79:0/2849102661 wait complete. 2024-04-26T19:51:25.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:25.251 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:25.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.350+0000 7fef4887e640 1 Processor -- start 2024-04-26T19:51:25.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.350+0000 7fef4887e640 1 -- start start 2024-04-26T19:51:25.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.351+0000 7fef4887e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef440b6630 0x7fef440b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.351+0000 7fef4887e640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fef4406c350 0x7fef4406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.351+0000 7fef4887e640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef440b7e60 con 0x7fef44148f30 2024-04-26T19:51:25.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.351+0000 7fef4887e640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fef4406ccf0 con 0x7fef4406c350 2024-04-26T19:51:25.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.351+0000 7fef4887e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fef440b6a70 con 0x7fef440b6630 2024-04-26T19:51:25.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.351+0000 7fef42d9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fef4406c350 0x7fef4406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.351+0000 7fef4259c640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fef44148f30 0x7fef440b7620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33358/0 (socket says 172.21.15.79:33358) 2024-04-26T19:51:25.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.351+0000 7fef4259c640 1 -- 172.21.15.79:0/371507163 learned_addr learned my addr 172.21.15.79:0/371507163 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef41d9b640 1 --2- 172.21.15.79:0/371507163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef440b6630 0x7fef440b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef42d9d640 1 -- 172.21.15.79:0/371507163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef440b6630 msgr2=0x7fef440b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef42d9d640 1 --2- 172.21.15.79:0/371507163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef440b6630 0x7fef440b8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef42d9d640 1 -- 172.21.15.79:0/371507163 >> v1:172.21.15.79:6789/0 conn(0x7fef44148f30 legacy=0x7fef440b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef4159a640 1 -- 172.21.15.79:0/371507163 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935709259 0 0) 0x7fef440b7e60 con 0x7fef44148f30 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef42d9d640 1 -- 172.21.15.79:0/371507163 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef440b8140 con 0x7fef4406c350 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef41d9b640 1 --2- 172.21.15.79:0/371507163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef440b6630 0x7fef440b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef42d9d640 1 --2- 172.21.15.79:0/371507163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef4406c350 0x7fef4406c7b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0097a0 tx=0x7fef3c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255f18a2771b3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef4159a640 1 -- 172.21.15.79:0/371507163 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c037070 con 0x7fef4406c350 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef4159a640 1 -- 172.21.15.79:0/371507163 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3c02ae50 con 0x7fef4406c350 2024-04-26T19:51:25.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.352+0000 7fef4159a640 1 -- 172.21.15.79:0/371507163 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3c032660 con 0x7fef4406c350 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.353+0000 7fef4887e640 1 -- 172.21.15.79:0/371507163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef4406c350 msgr2=0x7fef4406c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.353+0000 7fef4887e640 1 --2- 172.21.15.79:0/371507163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef4406c350 0x7fef4406c7b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0097a0 tx=0x7fef3c02a100 comp rx=0 tx=0).stop 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.353+0000 7fef4887e640 1 -- 172.21.15.79:0/371507163 shutdown_connections 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.353+0000 7fef4887e640 1 --2- 172.21.15.79:0/371507163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef4406c350 0x7fef4406c7b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.353+0000 7fef4887e640 1 --2- 172.21.15.79:0/371507163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef440b6630 0x7fef440b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.353+0000 7fef4887e640 1 -- 172.21.15.79:0/371507163 >> 172.21.15.79:0/371507163 conn(0x7fef44073b40 msgr2=0x7fef44073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.353+0000 7fef4887e640 1 -- 172.21.15.79:0/371507163 shutdown_connections 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.353+0000 7fef4887e640 1 -- 172.21.15.79:0/371507163 wait complete. 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.354+0000 7fef4887e640 1 Processor -- start 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.354+0000 7fef4887e640 1 -- start start 2024-04-26T19:51:25.358 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef4887e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef4406c350 0x7fef44144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef4887e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef440b6630 0x7fef4413e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef4887e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44148f30 0x7fef441f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef4887e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fef440b89e0 con 0x7fef440b6630 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef4887e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef440b8700 con 0x7fef44148f30 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef4887e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fef440b9440 con 0x7fef4406c350 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef41d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef440b6630 0x7fef4413e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef41d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef440b6630 0x7fef4413e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57296/0 (socket says 172.21.15.79:57296) 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef41d9b640 1 -- 172.21.15.79:0/2330378570 learned_addr learned my addr 172.21.15.79:0/2330378570 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef4259c640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef4406c350 0x7fef44144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef42d9d640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44148f30 0x7fef441f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef41d9b640 1 -- 172.21.15.79:0/2330378570 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef4406c350 msgr2=0x7fef44144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.355+0000 7fef41d9b640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef4406c350 0x7fef44144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef41d9b640 1 -- 172.21.15.79:0/2330378570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44148f30 msgr2=0x7fef441f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.359 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef41d9b640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44148f30 0x7fef441f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef41d9b640 1 -- 172.21.15.79:0/2330378570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4413ec10 con 0x7fef440b6630 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef42d9d640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44148f30 0x7fef441f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef41d9b640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef440b6630 0x7fef4413e6d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fef3800c7f0 tx=0x7fef3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef32ffd640 1 -- 172.21.15.79:0/2330378570 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38015070 con 0x7fef440b6630 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef32ffd640 1 -- 172.21.15.79:0/2330378570 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef38007a30 con 0x7fef440b6630 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef32ffd640 1 -- 172.21.15.79:0/2330378570 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38007e00 con 0x7fef440b6630 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef4413eec0 con 0x7fef440b6630 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.356+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef44141370 con 0x7fef440b6630 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.357+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef440506c0 con 0x7fef440b6630 2024-04-26T19:51:25.360 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.357+0000 7fef32ffd640 1 -- 172.21.15.79:0/2330378570 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef38018070 con 0x7fef440b6630 2024-04-26T19:51:25.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.358+0000 7fef32ffd640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fef2406c8f0 0x7fef2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.358+0000 7fef4259c640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fef2406c8f0 0x7fef2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.358+0000 7fef32ffd640 1 -- 172.21.15.79:0/2330378570 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fef3808bf20 con 0x7fef440b6630 2024-04-26T19:51:25.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.358+0000 7fef4259c640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fef2406c8f0 0x7fef2406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef34007ed0 tx=0x7fef34007e60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.362 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.362+0000 7fef32ffd640 1 -- 172.21.15.79:0/2330378570 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fef38059f40 con 0x7fef440b6630 2024-04-26T19:51:25.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.537+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fef441479b0 con 0x7fef440b6630 2024-04-26T19:51:25.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.538+0000 7fef32ffd640 1 -- 172.21.15.79:0/2330378570 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fef38059840 con 0x7fef440b6630 2024-04-26T19:51:25.538 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:25.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fef2406c8f0 msgr2=0x7fef2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fef2406c8f0 0x7fef2406edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fef34007ed0 tx=0x7fef34007e60 comp rx=0 tx=0).stop 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef440b6630 msgr2=0x7fef4413e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef440b6630 0x7fef4413e6d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fef3800c7f0 tx=0x7fef3800cbf0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 shutdown_connections 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fef2406c8f0 0x7fef2406edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef44148f30 0x7fef441f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef440b6630 0x7fef4413e6d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 --2- 172.21.15.79:0/2330378570 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef4406c350 0x7fef44144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 >> 172.21.15.79:0/2330378570 conn(0x7fef44073b40 msgr2=0x7fef440b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 shutdown_connections 2024-04-26T19:51:25.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.540+0000 7fef4887e640 1 -- 172.21.15.79:0/2330378570 wait complete. 2024-04-26T19:51:25.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:25.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.650+0000 7fad7fcc8640 1 Processor -- start 2024-04-26T19:51:25.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.651+0000 7fad7fcc8640 1 -- start start 2024-04-26T19:51:25.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.651+0000 7fad7fcc8640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fad780aeeb0 0x7fad7806f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.651+0000 7fad7fcc8640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fad78070160 0x7fad780b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.651+0000 7fad7fcc8640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 0x7fad780af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.651+0000 7fad7fcc8640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fad7806fa10 con 0x7fad780aeeb0 2024-04-26T19:51:25.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.651+0000 7fad7fcc8640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fad78072ea0 con 0x7fad78070160 2024-04-26T19:51:25.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.651+0000 7fad7fcc8640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fad78073180 con 0x7fad780b1990 2024-04-26T19:51:25.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.651+0000 7fad7da64640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fad780aeeb0 0x7fad7806f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7da64640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fad780aeeb0 0x7fad7806f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57300/0 (socket says 172.21.15.79:57300) 2024-04-26T19:51:25.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7da64640 1 -- 172.21.15.79:0/3112881762 learned_addr learned my addr 172.21.15.79:0/3112881762 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:25.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7d263640 1 --2- 172.21.15.79:0/3112881762 >> v2:172.21.15.83:3300/0 conn(0x7fad78070160 0x7fad780b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7e265640 1 --2- 172.21.15.79:0/3112881762 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 0x7fad780af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7da64640 1 -- 172.21.15.79:0/3112881762 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 msgr2=0x7fad780af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:25.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7da64640 1 --2- 172.21.15.79:0/3112881762 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 0x7fad780af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7da64640 1 -- 172.21.15.79:0/3112881762 >> v2:172.21.15.83:3300/0 conn(0x7fad78070160 msgr2=0x7fad780b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:25.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7da64640 1 --2- 172.21.15.79:0/3112881762 >> v2:172.21.15.83:3300/0 conn(0x7fad78070160 0x7fad780b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7da64640 1 -- 172.21.15.79:0/3112881762 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad78073460 con 0x7fad780aeeb0 2024-04-26T19:51:25.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.652+0000 7fad7da64640 1 --2- 172.21.15.79:0/3112881762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad780aeeb0 0x7fad7806f0e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fad780aee00 tx=0x7fad7402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bc1ef018397d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7ca62640 1 -- 172.21.15.79:0/3112881762 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74036070 con 0x7fad780aeeb0 2024-04-26T19:51:25.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7ca62640 1 -- 172.21.15.79:0/3112881762 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad74031600 con 0x7fad780aeeb0 2024-04-26T19:51:25.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7ca62640 1 -- 172.21.15.79:0/3112881762 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad740319d0 con 0x7fad780aeeb0 2024-04-26T19:51:25.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7fcc8640 1 -- 172.21.15.79:0/3112881762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad780aeeb0 msgr2=0x7fad7806f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/3112881762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad780aeeb0 0x7fad7806f0e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fad780aee00 tx=0x7fad7402a610 comp rx=0 tx=0).stop 2024-04-26T19:51:25.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7fcc8640 1 -- 172.21.15.79:0/3112881762 shutdown_connections 2024-04-26T19:51:25.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/3112881762 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 0x7fad780af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/3112881762 >> v2:172.21.15.83:3300/0 conn(0x7fad78070160 0x7fad780b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/3112881762 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad780aeeb0 0x7fad7806f0e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7fcc8640 1 -- 172.21.15.79:0/3112881762 >> 172.21.15.79:0/3112881762 conn(0x7fad780687e0 msgr2=0x7fad78071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:25.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7fcc8640 1 -- 172.21.15.79:0/3112881762 shutdown_connections 2024-04-26T19:51:25.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.653+0000 7fad7fcc8640 1 -- 172.21.15.79:0/3112881762 wait complete. 2024-04-26T19:51:25.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.654+0000 7fad7fcc8640 1 Processor -- start 2024-04-26T19:51:25.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.654+0000 7fad7fcc8640 1 -- start start 2024-04-26T19:51:25.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7fcc8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78070160 0x7fad780c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7fcc8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad780aeeb0 0x7fad7813f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7fcc8640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 0x7fad7814fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7fcc8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fad7806f620 con 0x7fad78070160 2024-04-26T19:51:25.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7fcc8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fad78149380 con 0x7fad780aeeb0 2024-04-26T19:51:25.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7fcc8640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fad7806f7a0 con 0x7fad780b1990 2024-04-26T19:51:25.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7da64640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78070160 0x7fad780c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7da64640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78070160 0x7fad780c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57302/0 (socket says 172.21.15.79:57302) 2024-04-26T19:51:25.665 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7da64640 1 -- 172.21.15.79:0/285967375 learned_addr learned my addr 172.21.15.79:0/285967375 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:25.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7d263640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad780aeeb0 0x7fad7813f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7e265640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 0x7fad7814fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7da64640 1 -- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 msgr2=0x7fad7814fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7da64640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 0x7fad7814fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7da64640 1 -- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad780aeeb0 msgr2=0x7fad7813f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7da64640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad780aeeb0 0x7fad7813f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7da64640 1 -- 172.21.15.79:0/285967375 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad78150340 con 0x7fad78070160 2024-04-26T19:51:25.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7d263640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad780aeeb0 0x7fad7813f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:25.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.655+0000 7fad7e265640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 0x7fad7814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:25.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.656+0000 7fad7da64640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78070160 0x7fad780c4e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad74034040 tx=0x7fad74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.656+0000 7fad6e7fc640 1 -- 172.21.15.79:0/285967375 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74036050 con 0x7fad78070160 2024-04-26T19:51:25.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.656+0000 7fad6e7fc640 1 -- 172.21.15.79:0/285967375 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad74042070 con 0x7fad78070160 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.656+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad78150610 con 0x7fad78070160 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.656+0000 7fad6e7fc640 1 -- 172.21.15.79:0/285967375 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74009510 con 0x7fad78070160 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.656+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad78150bd0 con 0x7fad78070160 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.657+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad780506c0 con 0x7fad78070160 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.657+0000 7fad6e7fc640 1 -- 172.21.15.79:0/285967375 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad74007c50 con 0x7fad78070160 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.658+0000 7fad6e7fc640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fad5c06c8f0 0x7fad5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.658+0000 7fad6e7fc640 1 -- 172.21.15.79:0/285967375 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fad740accc0 con 0x7fad78070160 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.658+0000 7fad7d263640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fad5c06c8f0 0x7fad5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.658+0000 7fad7d263640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fad5c06c8f0 0x7fad5c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fad781624d0 tx=0x7fad68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.662+0000 7fad6e7fc640 1 -- 172.21.15.79:0/285967375 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fad74045360 con 0x7fad78070160 2024-04-26T19:51:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.836+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fad7806a800 con 0x7fad78070160 2024-04-26T19:51:25.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.836+0000 7fad6e7fc640 1 -- 172.21.15.79:0/285967375 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fad7407ad80 con 0x7fad78070160 2024-04-26T19:51:25.837 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fad5c06c8f0 msgr2=0x7fad5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fad5c06c8f0 0x7fad5c06edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fad781624d0 tx=0x7fad68008040 comp rx=0 tx=0).stop 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78070160 msgr2=0x7fad780c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78070160 0x7fad780c4e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fad74034040 tx=0x7fad74004a40 comp rx=0 tx=0).stop 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 shutdown_connections 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fad5c06c8f0 0x7fad5c06edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fad780b1990 0x7fad7814fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fad780aeeb0 0x7fad7813f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 --2- 172.21.15.79:0/285967375 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fad78070160 0x7fad780c4e70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 >> 172.21.15.79:0/285967375 conn(0x7fad780687e0 msgr2=0x7fad7806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 shutdown_connections 2024-04-26T19:51:25.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.838+0000 7fad7fcc8640 1 -- 172.21.15.79:0/285967375 wait complete. 2024-04-26T19:51:25.850 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:25.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.949+0000 7fcd1e99e640 1 Processor -- start 2024-04-26T19:51:25.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.949+0000 7fcd1e99e640 1 -- start start 2024-04-26T19:51:25.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1e99e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd180d4a50 0x7fcd180ba4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1e99e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd180baa30 0x7fcd1806fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1e99e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd180bb490 con 0x7fcd1815f3e0 2024-04-26T19:51:25.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1e99e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fcd1815f7c0 con 0x7fcd180baa30 2024-04-26T19:51:25.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1e99e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fcd1806eec0 con 0x7fcd180d4a50 2024-04-26T19:51:25.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1d19b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd180d4a50 0x7fcd180ba4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1e19d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fcd180baa30 0x7fcd1806fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1d19b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd180d4a50 0x7fcd180ba4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:59992/0 (socket says 172.21.15.79:59992) 2024-04-26T19:51:25.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1d19b640 1 -- 172.21.15.79:0/3981471026 learned_addr learned my addr 172.21.15.79:0/3981471026 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:25.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1d99c640 1 --1- 172.21.15.79:0/3981471026 >> v1:172.21.15.83:6789/0 conn(0x7fcd1815f3e0 0x7fcd180d4340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:45882/0 (socket says 172.21.15.79:45882) 2024-04-26T19:51:25.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1d19b640 1 -- 172.21.15.79:0/3981471026 >> v2:172.21.15.83:3300/0 conn(0x7fcd180baa30 msgr2=0x7fcd1806fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:25.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1d19b640 1 --2- 172.21.15.79:0/3981471026 >> v2:172.21.15.83:3300/0 conn(0x7fcd180baa30 0x7fcd1806fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1d19b640 1 -- 172.21.15.79:0/3981471026 >> v1:172.21.15.83:6789/0 conn(0x7fcd1815f3e0 legacy=0x7fcd180d4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.950+0000 7fcd1d19b640 1 -- 172.21.15.79:0/3981471026 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd180bb1b0 con 0x7fcd180d4a50 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.951+0000 7fcd1d19b640 1 --2- 172.21.15.79:0/3981471026 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd180d4a50 0x7fcd180ba4f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009870 tx=0x7fcd0802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=821488561bb5bbd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.951+0000 7fcd1c99a640 1 -- 172.21.15.79:0/3981471026 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08037070 con 0x7fcd180d4a50 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.951+0000 7fcd1c99a640 1 -- 172.21.15.79:0/3981471026 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd08032440 con 0x7fcd180d4a50 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.951+0000 7fcd1c99a640 1 -- 172.21.15.79:0/3981471026 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08032810 con 0x7fcd180d4a50 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.952+0000 7fcd1e99e640 1 -- 172.21.15.79:0/3981471026 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd180d4a50 msgr2=0x7fcd180ba4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.952+0000 7fcd1e99e640 1 --2- 172.21.15.79:0/3981471026 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd180d4a50 0x7fcd180ba4f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009870 tx=0x7fcd0802a270 comp rx=0 tx=0).stop 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.952+0000 7fcd1e99e640 1 -- 172.21.15.79:0/3981471026 shutdown_connections 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.952+0000 7fcd1e99e640 1 --2- 172.21.15.79:0/3981471026 >> v2:172.21.15.83:3300/0 conn(0x7fcd180baa30 0x7fcd1806fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.952+0000 7fcd1e99e640 1 --2- 172.21.15.79:0/3981471026 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd180d4a50 0x7fcd180ba4f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.952+0000 7fcd1e99e640 1 -- 172.21.15.79:0/3981471026 >> 172.21.15.79:0/3981471026 conn(0x7fcd180d1f30 msgr2=0x7fcd180d2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.952+0000 7fcd1e99e640 1 -- 172.21.15.79:0/3981471026 shutdown_connections 2024-04-26T19:51:25.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.952+0000 7fcd1e99e640 1 -- 172.21.15.79:0/3981471026 wait complete. 2024-04-26T19:51:25.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.953+0000 7fcd1e99e640 1 Processor -- start 2024-04-26T19:51:25.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.953+0000 7fcd1e99e640 1 -- start start 2024-04-26T19:51:25.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1e99e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd180baa30 0x7fcd1814a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1e99e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd180d4a50 0x7fcd180b72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1e99e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd1815f3e0 0x7fcd180b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1e99e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1806f700 con 0x7fcd180baa30 2024-04-26T19:51:25.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1e99e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1806f420 con 0x7fcd180d4a50 2024-04-26T19:51:25.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1e99e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fcd180baec0 con 0x7fcd1815f3e0 2024-04-26T19:51:25.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d99c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd180baa30 0x7fcd1814a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d99c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd180baa30 0x7fcd1814a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57314/0 (socket says 172.21.15.79:57314) 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d99c640 1 -- 172.21.15.79:0/2312495611 learned_addr learned my addr 172.21.15.79:0/2312495611 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1e19d640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd1815f3e0 0x7fcd180b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d19b640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd180d4a50 0x7fcd180b72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d99c640 1 -- 172.21.15.79:0/2312495611 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd1815f3e0 msgr2=0x7fcd180b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d99c640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd1815f3e0 0x7fcd180b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d99c640 1 -- 172.21.15.79:0/2312495611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd180d4a50 msgr2=0x7fcd180b72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d99c640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd180d4a50 0x7fcd180b72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d99c640 1 -- 172.21.15.79:0/2312495611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1814a7f0 con 0x7fcd180baa30 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.954+0000 7fcd1d99c640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd180baa30 0x7fcd1814a2b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400c430 tx=0x7fcd1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.955+0000 7fcd0e7fc640 1 -- 172.21.15.79:0/2312495611 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd140074e0 con 0x7fcd180baa30 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.955+0000 7fcd0e7fc640 1 -- 172.21.15.79:0/2312495611 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd14007680 con 0x7fcd180baa30 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.955+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd18160ce0 con 0x7fcd180baa30 2024-04-26T19:51:25.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.955+0000 7fcd0e7fc640 1 -- 172.21.15.79:0/2312495611 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14004a00 con 0x7fcd180baa30 2024-04-26T19:51:25.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.955+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd18160210 con 0x7fcd180baa30 2024-04-26T19:51:25.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.956+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd18050f10 con 0x7fcd180baa30 2024-04-26T19:51:25.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.956+0000 7fcd0e7fc640 1 -- 172.21.15.79:0/2312495611 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd1401a020 con 0x7fcd180baa30 2024-04-26T19:51:25.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.957+0000 7fcd0e7fc640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fccfc06c8f0 0x7fccfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:25.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.957+0000 7fcd1d19b640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fccfc06c8f0 0x7fccfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:25.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.957+0000 7fcd0e7fc640 1 -- 172.21.15.79:0/2312495611 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcd1408b780 con 0x7fcd180baa30 2024-04-26T19:51:25.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.957+0000 7fcd1d19b640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fccfc06c8f0 0x7fccfc06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd08034000 tx=0x7fcd0802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:25.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:25.961+0000 7fcd0e7fc640 1 -- 172.21.15.79:0/2312495611 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fcd140597b0 con 0x7fcd180baa30 2024-04-26T19:51:26.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.137+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcd180d4430 con 0x7fcd180baa30 2024-04-26T19:51:26.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.137+0000 7fcd0e7fc640 1 -- 172.21.15.79:0/2312495611 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fcd140590b0 con 0x7fcd180baa30 2024-04-26T19:51:26.138 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:26.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.139+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fccfc06c8f0 msgr2=0x7fccfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.139+0000 7fcd1e99e640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fccfc06c8f0 0x7fccfc06edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcd08034000 tx=0x7fcd0802e000 comp rx=0 tx=0).stop 2024-04-26T19:51:26.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.139+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd180baa30 msgr2=0x7fcd1814a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.139+0000 7fcd1e99e640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd180baa30 0x7fcd1814a2b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400c430 tx=0x7fcd1400c8f0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.139+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 shutdown_connections 2024-04-26T19:51:26.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.139+0000 7fcd1e99e640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fccfc06c8f0 0x7fccfc06edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.139+0000 7fcd1e99e640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fcd1815f3e0 0x7fcd180b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.139+0000 7fcd1e99e640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fcd180d4a50 0x7fcd180b72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.139+0000 7fcd1e99e640 1 --2- 172.21.15.79:0/2312495611 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fcd180baa30 0x7fcd1814a2b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.140+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 >> 172.21.15.79:0/2312495611 conn(0x7fcd180d1f30 msgr2=0x7fcd1815d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:26.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.140+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 shutdown_connections 2024-04-26T19:51:26.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.140+0000 7fcd1e99e640 1 -- 172.21.15.79:0/2312495611 wait complete. 2024-04-26T19:51:26.151 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:26.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.250+0000 7f18e34bd640 1 Processor -- start 2024-04-26T19:51:26.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.250+0000 7f18e34bd640 1 -- start start 2024-04-26T19:51:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e34bd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f18dc1482e0 0x7f18dc0d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e34bd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f18dc148980 0x7f18dc06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e34bd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18dc0d2580 con 0x7f18dc06c890 2024-04-26T19:51:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e34bd640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f18dc0d2860 con 0x7f18dc148980 2024-04-26T19:51:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e34bd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f18dc0b6a80 con 0x7f18dc1482e0 2024-04-26T19:51:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e0a58640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f18dc148980 0x7f18dc06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e0a58640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f18dc148980 0x7f18dc06c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57320/0 (socket says 172.21.15.79:57320) 2024-04-26T19:51:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e0a58640 1 -- 172.21.15.79:0/2746750735 learned_addr learned my addr 172.21.15.79:0/2746750735 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:26.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e1259640 1 --2- 172.21.15.79:0/2746750735 >> v2:172.21.15.83:3300/0 conn(0x7f18dc1482e0 0x7f18dc0d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e0a58640 1 -- 172.21.15.79:0/2746750735 >> v2:172.21.15.83:3300/0 conn(0x7f18dc1482e0 msgr2=0x7f18dc0d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:26.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e0a58640 1 --2- 172.21.15.79:0/2746750735 >> v2:172.21.15.83:3300/0 conn(0x7f18dc1482e0 0x7f18dc0d1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e0a58640 1 -- 172.21.15.79:0/2746750735 >> v1:172.21.15.83:6789/0 conn(0x7f18dc06c890 legacy=0x7f18dc1631b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.251+0000 7f18e0a58640 1 -- 172.21.15.79:0/2746750735 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc0b6d60 con 0x7f18dc148980 2024-04-26T19:51:26.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.252+0000 7f18e0a58640 1 --2- 172.21.15.79:0/2746750735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc148980 0x7f18dc06c350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0d1060 tx=0x7f18cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6860711185be56de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:26.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.252+0000 7f18d3fff640 1 -- 172.21.15.79:0/2746750735 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc037070 con 0x7f18dc148980 2024-04-26T19:51:26.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.252+0000 7f18d3fff640 1 -- 172.21.15.79:0/2746750735 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18cc032440 con 0x7f18dc148980 2024-04-26T19:51:26.254 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.252+0000 7f18d3fff640 1 -- 172.21.15.79:0/2746750735 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18cc032810 con 0x7f18dc148980 2024-04-26T19:51:26.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.253+0000 7f18e34bd640 1 -- 172.21.15.79:0/2746750735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc148980 msgr2=0x7f18dc06c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.253+0000 7f18e34bd640 1 --2- 172.21.15.79:0/2746750735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc148980 0x7f18dc06c350 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0d1060 tx=0x7f18cc02a1e0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.253+0000 7f18e34bd640 1 -- 172.21.15.79:0/2746750735 shutdown_connections 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.253+0000 7f18e34bd640 1 --2- 172.21.15.79:0/2746750735 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc148980 0x7f18dc06c350 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.253+0000 7f18e34bd640 1 --2- 172.21.15.79:0/2746750735 >> v2:172.21.15.83:3300/0 conn(0x7f18dc1482e0 0x7f18dc0d1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.253+0000 7f18e34bd640 1 -- 172.21.15.79:0/2746750735 >> 172.21.15.79:0/2746750735 conn(0x7f18dc073b40 msgr2=0x7f18dc073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.253+0000 7f18e34bd640 1 -- 172.21.15.79:0/2746750735 shutdown_connections 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.253+0000 7f18e34bd640 1 -- 172.21.15.79:0/2746750735 wait complete. 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.254+0000 7f18e34bd640 1 Processor -- start 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.254+0000 7f18e34bd640 1 -- start start 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e34bd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc06c890 0x7f18dc15f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e34bd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc1482e0 0x7f18dc1f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e34bd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f18dc148980 0x7f18dc06f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e34bd640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc149100 con 0x7f18dc06c890 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e34bd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc1486c0 con 0x7f18dc1482e0 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e34bd640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f18dc0b8290 con 0x7f18dc148980 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e1259640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc06c890 0x7f18dc15f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e1259640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc06c890 0x7f18dc15f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57324/0 (socket says 172.21.15.79:57324) 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e1259640 1 -- 172.21.15.79:0/1664384648 learned_addr learned my addr 172.21.15.79:0/1664384648 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e1259640 1 -- 172.21.15.79:0/1664384648 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f18dc148980 msgr2=0x7f18dc06f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e1259640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f18dc148980 0x7f18dc06f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e1259640 1 -- 172.21.15.79:0/1664384648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc1482e0 msgr2=0x7f18dc1f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e1259640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc1482e0 0x7f18dc1f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e1259640 1 -- 172.21.15.79:0/1664384648 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc0ccf00 con 0x7f18dc06c890 2024-04-26T19:51:26.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e1259640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc06c890 0x7f18dc15f020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f18d800efc0 tx=0x7f18d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:26.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18d1ffb640 1 -- 172.21.15.79:0/1664384648 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8018020 con 0x7f18dc06c890 2024-04-26T19:51:26.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18d1ffb640 1 -- 172.21.15.79:0/1664384648 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18d8014480 con 0x7f18dc06c890 2024-04-26T19:51:26.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc1f9090 con 0x7f18dc06c890 2024-04-26T19:51:26.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.255+0000 7f18d1ffb640 1 -- 172.21.15.79:0/1664384648 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18d8014850 con 0x7f18dc06c890 2024-04-26T19:51:26.259 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.256+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1f9650 con 0x7f18dc06c890 2024-04-26T19:51:26.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.257+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18dc0b93a0 con 0x7f18dc06c890 2024-04-26T19:51:26.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.257+0000 7f18d1ffb640 1 -- 172.21.15.79:0/1664384648 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f18d8005d10 con 0x7f18dc06c890 2024-04-26T19:51:26.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.257+0000 7f18d1ffb640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f18c006c8f0 0x7f18c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.258+0000 7f18d1ffb640 1 -- 172.21.15.79:0/1664384648 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f18d808ca10 con 0x7f18dc06c890 2024-04-26T19:51:26.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.258+0000 7f18e0a58640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f18c006c8f0 0x7f18c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.258+0000 7f18e0a58640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f18c006c8f0 0x7f18c006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f18cc034000 tx=0x7f18cc02c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:26.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.262+0000 7f18d1ffb640 1 -- 172.21.15.79:0/1664384648 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f18dc0b93a0 con 0x7f18dc06c890 2024-04-26T19:51:26.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.440+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f18dc06c6b0 con 0x7f18dc06c890 2024-04-26T19:51:26.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.441+0000 7f18d1ffb640 1 -- 172.21.15.79:0/1664384648 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f18dc06c6b0 con 0x7f18dc06c890 2024-04-26T19:51:26.442 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:26.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f18c006c8f0 msgr2=0x7f18c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f18c006c8f0 0x7f18c006edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f18cc034000 tx=0x7f18cc02c000 comp rx=0 tx=0).stop 2024-04-26T19:51:26.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc06c890 msgr2=0x7f18dc15f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc06c890 0x7f18dc15f020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f18d800efc0 tx=0x7f18d800c490 comp rx=0 tx=0).stop 2024-04-26T19:51:26.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 shutdown_connections 2024-04-26T19:51:26.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f18c006c8f0 0x7f18c006edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f18dc148980 0x7f18dc06f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f18dc1482e0 0x7f18dc1f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 --2- 172.21.15.79:0/1664384648 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f18dc06c890 0x7f18dc15f020 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 >> 172.21.15.79:0/1664384648 conn(0x7f18dc073b40 msgr2=0x7f18dc1625a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:26.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 shutdown_connections 2024-04-26T19:51:26.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.443+0000 7f18e34bd640 1 -- 172.21.15.79:0/1664384648 wait complete. 2024-04-26T19:51:26.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T19:51:26.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.552+0000 7f3cc3b89640 1 Processor -- start 2024-04-26T19:51:26.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.552+0000 7f3cc3b89640 1 -- start start 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc3b89640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc0b64f0 0x7f3cbc070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc3b89640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cbc06e800 0x7f3cbc0b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc3b89640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cbc15f810 con 0x7f3cbc15ffa0 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc3b89640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3cbc15fb10 con 0x7f3cbc06e800 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc3b89640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3cbc06f7a0 con 0x7f3cbc0b64f0 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc2126640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3cbc15ffa0 0x7f3cbc160a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33366/0 (socket says 172.21.15.79:33366) 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc2126640 1 -- 172.21.15.79:0/1277069926 learned_addr learned my addr 172.21.15.79:0/1277069926 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902486311 0 0) 0x7f3cbc15f810 con 0x7f3cbc15ffa0 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ca8003520 con 0x7f3cbc15ffa0 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc1925640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc0b64f0 0x7f3cbc070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc1124640 1 --2- 172.21.15.79:0/1277069926 >> v2:172.21.15.83:3300/0 conn(0x7f3cbc06e800 0x7f3cbc0b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1223420908 0 0) 0x7f3ca8003520 con 0x7f3cbc15ffa0 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc0b64f0 msgr2=0x7f3cbc070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:26.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc0b64f0 0x7f3cbc070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 >> v2:172.21.15.83:3300/0 conn(0x7f3cbc06e800 msgr2=0x7f3cbc0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 --2- 172.21.15.79:0/1277069926 >> v2:172.21.15.83:3300/0 conn(0x7f3cbc06e800 0x7f3cbc0b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cbc06ec60 con 0x7f3cbc15ffa0 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f3cb40019f0 con 0x7f3cbc15ffa0 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc1124640 1 --2- 172.21.15.79:0/1277069926 >> v2:172.21.15.83:3300/0 conn(0x7f3cbc06e800 0x7f3cbc0b68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.553+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 >> v1:172.21.15.79:6789/0 conn(0x7f3cbc15ffa0 legacy=0x7f3cbc160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc0923640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc0923640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc0923640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc0b64f0 0x7f3ca80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc15fb10 con 0x7f3ca80039b0 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc06f7a0 con 0x7f3ca8004800 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca800b9c0 con 0x7f3cbc0b64f0 2024-04-26T19:51:26.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc1925640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc1124640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc2126640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc0b64f0 0x7f3ca80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc1925640 1 -- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc0b64f0 msgr2=0x7f3ca80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc1925640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc0b64f0 0x7f3ca80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc1925640 1 -- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ca8004800 msgr2=0x7f3ca8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc1925640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc1925640 1 -- 172.21.15.79:0/1277069926 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca8003520 con 0x7f3ca80039b0 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc1124640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3ca8004800 0x7f3ca8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.554+0000 7f3cc1925640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009ac0 tx=0x7f3ca8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2358acae983ca43 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8036070 con 0x7f3ca80039b0 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb8031400 con 0x7f3ca80039b0 2024-04-26T19:51:26.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc0923640 1 -- 172.21.15.79:0/1277069926 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8009690 con 0x7f3ca80039b0 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc3b89640 1 -- 172.21.15.79:0/1277069926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 msgr2=0x7f3ca8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc3b89640 1 --2- 172.21.15.79:0/1277069926 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3ca8003dd0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009ac0 tx=0x7f3ca8005830 comp rx=0 tx=0).stop 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc2126640 1 -- 172.21.15.79:0/1277069926 reap_dead start 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc3b89640 1 -- 172.21.15.79:0/1277069926 shutdown_connections 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc3b89640 1 -- 172.21.15.79:0/1277069926 >> 172.21.15.79:0/1277069926 conn(0x7f3cbc06c770 msgr2=0x7f3cbc06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc3b89640 1 -- 172.21.15.79:0/1277069926 shutdown_connections 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.555+0000 7f3cc3b89640 1 -- 172.21.15.79:0/1277069926 wait complete. 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.556+0000 7f3cc3b89640 1 Processor -- start 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.556+0000 7f3cc3b89640 1 -- start start 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc3b89640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3cbc1446a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc3b89640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc14fc00 0x7f3cbc150080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc3b89640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc152100 0x7f3cbc152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc3b89640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc06f1e0 con 0x7f3ca80039b0 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc3b89640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cbc06ef00 con 0x7f3cbc152100 2024-04-26T19:51:26.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc3b89640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3cbc06f4c0 con 0x7f3cbc14fc00 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc1925640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3cbc1446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc1925640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3cbc1446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57348/0 (socket says 172.21.15.79:57348) 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc1925640 1 -- 172.21.15.79:0/2314875830 learned_addr learned my addr 172.21.15.79:0/2314875830 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc1124640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc14fc00 0x7f3cbc150080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc2126640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc152100 0x7f3cbc152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc1925640 1 -- 172.21.15.79:0/2314875830 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc14fc00 msgr2=0x7f3cbc150080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc1925640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc14fc00 0x7f3cbc150080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc1925640 1 -- 172.21.15.79:0/2314875830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc152100 msgr2=0x7f3cbc152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc1925640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc152100 0x7f3cbc152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.557+0000 7f3cc1925640 1 -- 172.21.15.79:0/2314875830 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cbc150960 con 0x7f3ca80039b0 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.558+0000 7f3cc2126640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc152100 0x7f3cbc152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.558+0000 7f3cc1925640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3cbc1446a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009ac0 tx=0x7f3cb8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.558+0000 7f3cb27fc640 1 -- 172.21.15.79:0/2314875830 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8040020 con 0x7f3ca80039b0 2024-04-26T19:51:26.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.558+0000 7f3cb27fc640 1 -- 172.21.15.79:0/2314875830 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3cb8003bc0 con 0x7f3ca80039b0 2024-04-26T19:51:26.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.558+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cbc151620 con 0x7f3ca80039b0 2024-04-26T19:51:26.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.558+0000 7f3cb27fc640 1 -- 172.21.15.79:0/2314875830 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8036040 con 0x7f3ca80039b0 2024-04-26T19:51:26.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.558+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cbc144be0 con 0x7f3ca80039b0 2024-04-26T19:51:26.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.559+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cbc0506c0 con 0x7f3ca80039b0 2024-04-26T19:51:26.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.559+0000 7f3cb27fc640 1 -- 172.21.15.79:0/2314875830 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3cb8043020 con 0x7f3ca80039b0 2024-04-26T19:51:26.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.560+0000 7f3cb27fc640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ca006c8f0 0x7f3ca006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:26.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.560+0000 7f3cb27fc640 1 -- 172.21.15.79:0/2314875830 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3cb80abfe0 con 0x7f3ca80039b0 2024-04-26T19:51:26.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.560+0000 7f3cc1124640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ca006c8f0 0x7f3ca006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:26.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.560+0000 7f3cc1124640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ca006c8f0 0x7f3ca006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0b6d50 tx=0x7f3cac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:26.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.564+0000 7f3cb27fc640 1 -- 172.21.15.79:0/2314875830 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3cb804a070 con 0x7f3ca80039b0 2024-04-26T19:51:26.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.751+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3cbc15f840 con 0x7f3ca80039b0 2024-04-26T19:51:26.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.751+0000 7f3cb27fc640 1 -- 172.21.15.79:0/2314875830 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f3cb80044e0 con 0x7f3ca80039b0 2024-04-26T19:51:26.752 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2024-04-26T19:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ca006c8f0 msgr2=0x7f3ca006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ca006c8f0 0x7f3ca006edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0b6d50 tx=0x7f3cac008040 comp rx=0 tx=0).stop 2024-04-26T19:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 msgr2=0x7f3cbc1446a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3cbc1446a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8009ac0 tx=0x7f3cb8004730 comp rx=0 tx=0).stop 2024-04-26T19:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 shutdown_connections 2024-04-26T19:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ca006c8f0 0x7f3ca006edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cbc152100 0x7f3cbc152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cbc14fc00 0x7f3cbc150080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 --2- 172.21.15.79:0/2314875830 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3ca80039b0 0x7f3cbc1446a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 >> 172.21.15.79:0/2314875830 conn(0x7f3cbc06c770 msgr2=0x7f3cbc15d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 shutdown_connections 2024-04-26T19:51:26.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:26.753+0000 7f3cc3b89640 1 -- 172.21.15.79:0/2314875830 wait complete. 2024-04-26T19:51:26.764 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-26T19:51:26.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:26.765 DEBUG:teuthology.orchestra.run.smithi079:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-26T19:51:26.789 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:51:26.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:26.789 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-26T19:51:26.845 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:51:26.845 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:26.864 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:51:26.865 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:26.922 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:51:26.923 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:26.984 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.078+0000 7fe15f99d640 1 Processor -- start 2024-04-26T19:51:27.079 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.078+0000 7fe15f99d640 1 -- start start 2024-04-26T19:51:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.079+0000 7fe15f99d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe15815c6e0 0x7fe15815cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.079+0000 7fe15f99d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe1580d05e0 0x7fe158165c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.079+0000 7fe15f99d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe15815d8d0 con 0x7fe15815d000 2024-04-26T19:51:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.079+0000 7fe15f99d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe15815dbb0 con 0x7fe15815c6e0 2024-04-26T19:51:27.080 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.079+0000 7fe15f99d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe15815de90 con 0x7fe1580d05e0 2024-04-26T19:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.080+0000 7fe15df3a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe1580d05e0 0x7fe158165c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.080+0000 7fe15d739640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe15815c6e0 0x7fe15815cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.080+0000 7fe15df3a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe1580d05e0 0x7fe158165c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:60046/0 (socket says 172.21.15.79:60046) 2024-04-26T19:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.080+0000 7fe15df3a640 1 -- 172.21.15.79:0/863954680 learned_addr learned my addr 172.21.15.79:0/863954680 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.080+0000 7fe15df3a640 1 -- 172.21.15.79:0/863954680 >> v2:172.21.15.83:3300/0 conn(0x7fe15815c6e0 msgr2=0x7fe15815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.080+0000 7fe15df3a640 1 --2- 172.21.15.79:0/863954680 >> v2:172.21.15.83:3300/0 conn(0x7fe15815c6e0 0x7fe15815cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.080+0000 7fe15df3a640 1 -- 172.21.15.79:0/863954680 >> v1:172.21.15.83:6789/0 conn(0x7fe15815d000 legacy=0x7fe1580cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.080+0000 7fe15df3a640 1 -- 172.21.15.79:0/863954680 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15815e170 con 0x7fe1580d05e0 2024-04-26T19:51:27.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.080+0000 7fe15d739640 1 --2- 172.21.15.79:0/863954680 >> v2:172.21.15.83:3300/0 conn(0x7fe15815c6e0 0x7fe15815cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:27.082 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.081+0000 7fe15df3a640 1 --2- 172.21.15.79:0/863954680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe1580d05e0 0x7fe158165c90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe154009870 tx=0x7fe15402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0509814f5221b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.081+0000 7fe14ffff640 1 -- 172.21.15.79:0/863954680 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154037070 con 0x7fe1580d05e0 2024-04-26T19:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.081+0000 7fe14ffff640 1 -- 172.21.15.79:0/863954680 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe154032440 con 0x7fe1580d05e0 2024-04-26T19:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.081+0000 7fe14ffff640 1 -- 172.21.15.79:0/863954680 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154032810 con 0x7fe1580d05e0 2024-04-26T19:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.082+0000 7fe15f99d640 1 -- 172.21.15.79:0/863954680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe1580d05e0 msgr2=0x7fe158165c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.082+0000 7fe15f99d640 1 --2- 172.21.15.79:0/863954680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe1580d05e0 0x7fe158165c90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe154009870 tx=0x7fe15402a270 comp rx=0 tx=0).stop 2024-04-26T19:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.082+0000 7fe15f99d640 1 -- 172.21.15.79:0/863954680 shutdown_connections 2024-04-26T19:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.082+0000 7fe15f99d640 1 --2- 172.21.15.79:0/863954680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe1580d05e0 0x7fe158165c90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.082+0000 7fe15f99d640 1 --2- 172.21.15.79:0/863954680 >> v2:172.21.15.83:3300/0 conn(0x7fe15815c6e0 0x7fe15815cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.082+0000 7fe15f99d640 1 -- 172.21.15.79:0/863954680 >> 172.21.15.79:0/863954680 conn(0x7fe158073610 msgr2=0x7fe15806c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.082+0000 7fe15f99d640 1 -- 172.21.15.79:0/863954680 shutdown_connections 2024-04-26T19:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.082+0000 7fe15f99d640 1 -- 172.21.15.79:0/863954680 wait complete. 2024-04-26T19:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.083+0000 7fe15f99d640 1 Processor -- start 2024-04-26T19:51:27.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.083+0000 7fe15f99d640 1 -- start start 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15f99d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1580d05e0 0x7fe1581f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15f99d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe15815c6e0 0x7fe1580720e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15f99d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe15815d000 0x7fe158072620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15f99d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe15815f980 con 0x7fe15815d000 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15f99d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe15815a2b0 con 0x7fe1580d05e0 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15f99d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe158156aa0 con 0x7fe15815c6e0 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15df3a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe15815d000 0x7fe158072620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15df3a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe15815d000 0x7fe158072620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57352/0 (socket says 172.21.15.79:57352) 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15df3a640 1 -- 172.21.15.79:0/3886247253 learned_addr learned my addr 172.21.15.79:0/3886247253 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15d739640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1580d05e0 0x7fe1581f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15cf38640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe15815c6e0 0x7fe1580720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15df3a640 1 -- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe15815c6e0 msgr2=0x7fe1580720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15df3a640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe15815c6e0 0x7fe1580720e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15df3a640 1 -- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1580d05e0 msgr2=0x7fe1581f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15df3a640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1580d05e0 0x7fe1581f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15df3a640 1 -- 172.21.15.79:0/3886247253 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15814b490 con 0x7fe15815d000 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15d739640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1580d05e0 0x7fe1581f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15cf38640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe15815c6e0 0x7fe1580720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15df3a640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe15815d000 0x7fe158072620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe15402a2b0 tx=0x7fe1540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe14dffb640 1 -- 172.21.15.79:0/3886247253 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154041020 con 0x7fe15815d000 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe14dffb640 1 -- 172.21.15.79:0/3886247253 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe15403d640 con 0x7fe15815d000 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe14dffb640 1 -- 172.21.15.79:0/3886247253 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe154037040 con 0x7fe15815d000 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.084+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1581598a0 con 0x7fe15815d000 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.085+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581f8ba0 con 0x7fe15815d000 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.086+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1580506c0 con 0x7fe15815d000 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.086+0000 7fe14dffb640 1 -- 172.21.15.79:0/3886247253 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe154044020 con 0x7fe15815d000 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.086+0000 7fe14dffb640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe13c06c8f0 0x7fe13c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.086+0000 7fe14dffb640 1 -- 172.21.15.79:0/3886247253 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe1540ac630 con 0x7fe15815d000 2024-04-26T19:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.087+0000 7fe15d739640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe13c06c8f0 0x7fe13c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.087+0000 7fe15d739640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe13c06c8f0 0x7fe13c06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe1500074d0 tx=0x7fe15000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.091+0000 7fe14dffb640 1 -- 172.21.15.79:0/3886247253 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe15404b030 con 0x7fe15815d000 2024-04-26T19:51:27.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.271+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1580cea80 con 0x7fe15815d000 2024-04-26T19:51:27.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.272+0000 7fe14dffb640 1 -- 172.21.15.79:0/3886247253 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7fe15407a670 con 0x7fe15815d000 2024-04-26T19:51:27.273 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:27.274 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:27.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe13c06c8f0 msgr2=0x7fe13c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe13c06c8f0 0x7fe13c06edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe1500074d0 tx=0x7fe15000b040 comp rx=0 tx=0).stop 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe15815d000 msgr2=0x7fe158072620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe15815d000 0x7fe158072620 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fe15402a2b0 tx=0x7fe1540046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 shutdown_connections 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe13c06c8f0 0x7fe13c06edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe15815d000 0x7fe158072620 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe15815c6e0 0x7fe1580720e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 --2- 172.21.15.79:0/3886247253 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe1580d05e0 0x7fe1581f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 >> 172.21.15.79:0/3886247253 conn(0x7fe158073610 msgr2=0x7fe158070720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 shutdown_connections 2024-04-26T19:51:27.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.275+0000 7fe15f99d640 1 -- 172.21.15.79:0/3886247253 wait complete. 2024-04-26T19:51:27.287 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:27.288 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:27.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.387+0000 7fb9717e2640 1 Processor -- start 2024-04-26T19:51:27.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.387+0000 7fb9717e2640 1 -- start start 2024-04-26T19:51:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.388+0000 7fb9717e2640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0cfea0 0x7fb96c15d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.388+0000 7fb9717e2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb96c0d1380 0x7fb96c161430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.388+0000 7fb9717e2640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb96c0b8c70 con 0x7fb96c0b6f40 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.388+0000 7fb9717e2640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb96c157e50 con 0x7fb96c0d1380 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.388+0000 7fb9717e2640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb96c160fa0 con 0x7fb96c0cfea0 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.388+0000 7fb96affd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0cfea0 0x7fb96c15d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.388+0000 7fb96b7fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb96c0d1380 0x7fb96c161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.388+0000 7fb96affd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0cfea0 0x7fb96c15d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:60062/0 (socket says 172.21.15.79:60062) 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.388+0000 7fb96affd640 1 -- 172.21.15.79:0/470774471 learned_addr learned my addr 172.21.15.79:0/470774471 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.389+0000 7fb96b7fe640 1 -- 172.21.15.79:0/470774471 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0cfea0 msgr2=0x7fb96c15d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.389+0000 7fb96b7fe640 1 --2- 172.21.15.79:0/470774471 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0cfea0 0x7fb96c15d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.389+0000 7fb96b7fe640 1 -- 172.21.15.79:0/470774471 >> v1:172.21.15.83:6789/0 conn(0x7fb96c0b6f40 legacy=0x7fb96c0d0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.389+0000 7fb96b7fe640 1 -- 172.21.15.79:0/470774471 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c0b73c0 con 0x7fb96c0d1380 2024-04-26T19:51:27.390 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.389+0000 7fb96affd640 1 --2- 172.21.15.79:0/470774471 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0cfea0 0x7fb96c15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.389+0000 7fb96b7fe640 1 --2- 172.21.15.79:0/470774471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 0x7fb96c161430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb9640097a0 tx=0x7fb96402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94418f72e75db3dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb969ffb640 1 -- 172.21.15.79:0/470774471 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964037070 con 0x7fb96c0d1380 2024-04-26T19:51:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb969ffb640 1 -- 172.21.15.79:0/470774471 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96402ae50 con 0x7fb96c0d1380 2024-04-26T19:51:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb969ffb640 1 -- 172.21.15.79:0/470774471 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb964032660 con 0x7fb96c0d1380 2024-04-26T19:51:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb9717e2640 1 -- 172.21.15.79:0/470774471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 msgr2=0x7fb96c161430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb9717e2640 1 --2- 172.21.15.79:0/470774471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 0x7fb96c161430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb9640097a0 tx=0x7fb96402a100 comp rx=0 tx=0).stop 2024-04-26T19:51:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb9717e2640 1 -- 172.21.15.79:0/470774471 shutdown_connections 2024-04-26T19:51:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb9717e2640 1 --2- 172.21.15.79:0/470774471 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 0x7fb96c161430 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb9717e2640 1 --2- 172.21.15.79:0/470774471 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0cfea0 0x7fb96c15d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb9717e2640 1 -- 172.21.15.79:0/470774471 >> 172.21.15.79:0/470774471 conn(0x7fb96c073b40 msgr2=0x7fb96c073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb9717e2640 1 -- 172.21.15.79:0/470774471 shutdown_connections 2024-04-26T19:51:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.390+0000 7fb9717e2640 1 -- 172.21.15.79:0/470774471 wait complete. 2024-04-26T19:51:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.391+0000 7fb9717e2640 1 Processor -- start 2024-04-26T19:51:27.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.391+0000 7fb9717e2640 1 -- start start 2024-04-26T19:51:27.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb9717e2640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0b6f40 0x7fb96c15a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb9717e2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c0cfea0 0x7fb96c13c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb9717e2640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 0x7fb96c13c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb9717e2640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c15e4f0 con 0x7fb96c0cfea0 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb9717e2640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c15e210 con 0x7fb96c0d1380 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb9717e2640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb96c15e7d0 con 0x7fb96c0b6f40 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb96a7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c0cfea0 0x7fb96c13c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb96a7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c0cfea0 0x7fb96c13c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:57364/0 (socket says 172.21.15.79:57364) 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb96a7fc640 1 -- 172.21.15.79:0/2807876525 learned_addr learned my addr 172.21.15.79:0/2807876525 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb96b7fe640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 0x7fb96c13c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.392+0000 7fb96affd640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0b6f40 0x7fb96c15a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb96a7fc640 1 -- 172.21.15.79:0/2807876525 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0b6f40 msgr2=0x7fb96c15a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb96a7fc640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0b6f40 0x7fb96c15a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb96a7fc640 1 -- 172.21.15.79:0/2807876525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 msgr2=0x7fb96c13c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb96a7fc640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 0x7fb96c13c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb96a7fc640 1 -- 172.21.15.79:0/2807876525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c13a480 con 0x7fb96c0cfea0 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb96b7fe640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 0x7fb96c13c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb96a7fc640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c0cfea0 0x7fb96c13c420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb96000c430 tx=0x7fb96000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb9537fe640 1 -- 172.21.15.79:0/2807876525 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9600052d0 con 0x7fb96c0cfea0 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb9537fe640 1 -- 172.21.15.79:0/2807876525 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb960005490 con 0x7fb96c0cfea0 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c13a760 con 0x7fb96c0cfea0 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb9537fe640 1 -- 172.21.15.79:0/2807876525 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb960007260 con 0x7fb96c0cfea0 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.393+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c13ad20 con 0x7fb96c0cfea0 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.394+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c0506c0 con 0x7fb96c0cfea0 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.394+0000 7fb9537fe640 1 -- 172.21.15.79:0/2807876525 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb960007400 con 0x7fb96c0cfea0 2024-04-26T19:51:27.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.395+0000 7fb9537fe640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb95806c920 0x7fb95806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.395+0000 7fb9537fe640 1 -- 172.21.15.79:0/2807876525 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb96008b500 con 0x7fb96c0cfea0 2024-04-26T19:51:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.395+0000 7fb96affd640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb95806c920 0x7fb95806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.395+0000 7fb96affd640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb95806c920 0x7fb95806ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009240 tx=0x7fb95c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.399+0000 7fb9537fe640 1 -- 172.21.15.79:0/2807876525 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb96c0b8c70 con 0x7fb96c0cfea0 2024-04-26T19:51:27.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.575+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb96c0b40b0 con 0x7fb96c0cfea0 2024-04-26T19:51:27.577 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.575+0000 7fb9537fe640 1 -- 172.21.15.79:0/2807876525 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fb9600594b0 con 0x7fb96c0cfea0 2024-04-26T19:51:27.577 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:27.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb95806c920 msgr2=0x7fb95806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb95806c920 0x7fb95806ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb95c009240 tx=0x7fb95c00b040 comp rx=0 tx=0).stop 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c0cfea0 msgr2=0x7fb96c13c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c0cfea0 0x7fb96c13c420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb96000c430 tx=0x7fb96000c8f0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 shutdown_connections 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb95806c920 0x7fb95806ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb96c0d1380 0x7fb96c13c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb96c0cfea0 0x7fb96c13c420 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 --2- 172.21.15.79:0/2807876525 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb96c0b6f40 0x7fb96c15a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 >> 172.21.15.79:0/2807876525 conn(0x7fb96c073b40 msgr2=0x7fb96c0d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 shutdown_connections 2024-04-26T19:51:27.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.577+0000 7fb9717e2640 1 -- 172.21.15.79:0/2807876525 wait complete. 2024-04-26T19:51:27.590 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:27.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.689+0000 7f1f42acf640 1 Processor -- start 2024-04-26T19:51:27.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.689+0000 7f1f42acf640 1 -- start start 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f42acf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f3c0b6630 0x7f1f3c0b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f42acf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 0x7f1f3c06c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f42acf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f3c0b7e60 con 0x7f1f3c148f30 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f42acf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1f3c0b6a70 con 0x7f1f3c0b6630 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f42acf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3c0b8140 con 0x7f1f3c06c350 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f4106c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 0x7f1f3c06c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f3bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f3c0b6630 0x7f1f3c0b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f3bfff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1f3c0b6630 0x7f1f3c0b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:49334/0 (socket says 172.21.15.79:49334) 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f4106c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 0x7f1f3c06c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:37484/0 (socket says 172.21.15.79:37484) 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f3bfff640 1 -- 172.21.15.79:0/1341483949 learned_addr learned my addr 172.21.15.79:0/1341483949 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:27.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f4086b640 1 --1- 172.21.15.79:0/1341483949 >> v1:172.21.15.83:6789/0 conn(0x7f1f3c148f30 0x7f1f3c0b7620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:44090/0 (socket says 172.21.15.79:44090) 2024-04-26T19:51:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f3bfff640 1 -- 172.21.15.79:0/1341483949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 msgr2=0x7f1f3c06c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f3bfff640 1 --2- 172.21.15.79:0/1341483949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 0x7f1f3c06c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f3bfff640 1 -- 172.21.15.79:0/1341483949 >> v1:172.21.15.83:6789/0 conn(0x7f1f3c148f30 legacy=0x7f1f3c0b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.692 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.690+0000 7f1f3bfff640 1 -- 172.21.15.79:0/1341483949 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c0b8420 con 0x7f1f3c0b6630 2024-04-26T19:51:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.691+0000 7f1f3bfff640 1 --2- 172.21.15.79:0/1341483949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f3c0b6630 0x7f1f3c0b8f00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009870 tx=0x7f1f2c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bced656c17a9374 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.691+0000 7f1f3b7fe640 1 -- 172.21.15.79:0/1341483949 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c037070 con 0x7f1f3c0b6630 2024-04-26T19:51:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.691+0000 7f1f3b7fe640 1 -- 172.21.15.79:0/1341483949 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f2c032440 con 0x7f1f3c0b6630 2024-04-26T19:51:27.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.691+0000 7f1f3b7fe640 1 -- 172.21.15.79:0/1341483949 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c032810 con 0x7f1f3c0b6630 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.692+0000 7f1f42acf640 1 -- 172.21.15.79:0/1341483949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f3c0b6630 msgr2=0x7f1f3c0b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.692+0000 7f1f42acf640 1 --2- 172.21.15.79:0/1341483949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f3c0b6630 0x7f1f3c0b8f00 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009870 tx=0x7f1f2c02a1c0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.692+0000 7f1f42acf640 1 -- 172.21.15.79:0/1341483949 shutdown_connections 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.692+0000 7f1f42acf640 1 --2- 172.21.15.79:0/1341483949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 0x7f1f3c06c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.692+0000 7f1f42acf640 1 --2- 172.21.15.79:0/1341483949 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f3c0b6630 0x7f1f3c0b8f00 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.692+0000 7f1f42acf640 1 -- 172.21.15.79:0/1341483949 >> 172.21.15.79:0/1341483949 conn(0x7f1f3c073b40 msgr2=0x7f1f3c073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.692+0000 7f1f42acf640 1 -- 172.21.15.79:0/1341483949 shutdown_connections 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.692+0000 7f1f42acf640 1 -- 172.21.15.79:0/1341483949 wait complete. 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.693+0000 7f1f42acf640 1 Processor -- start 2024-04-26T19:51:27.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.693+0000 7f1f42acf640 1 -- start start 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f42acf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 0x7f1f3c1f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f42acf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f3c0b6630 0x7f1f3c13fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f42acf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f3c148f30 0x7f1f3c1627b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f42acf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c0b9440 con 0x7f1f3c148f30 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f42acf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c0b89e0 con 0x7f1f3c0b6630 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f42acf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1f3c15fbf0 con 0x7f1f3c06c350 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4106c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f3c148f30 0x7f1f3c1627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4106c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f3c148f30 0x7f1f3c1627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34902/0 (socket says 172.21.15.79:34902) 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4106c640 1 -- 172.21.15.79:0/4134198109 learned_addr learned my addr 172.21.15.79:0/4134198109 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f3bfff640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f3c0b6630 0x7f1f3c13fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4086b640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 0x7f1f3c1f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4106c640 1 -- 172.21.15.79:0/4134198109 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 msgr2=0x7f1f3c1f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4106c640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 0x7f1f3c1f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.697 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4106c640 1 -- 172.21.15.79:0/4134198109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f3c0b6630 msgr2=0x7f1f3c13fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4106c640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f3c0b6630 0x7f1f3c13fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4106c640 1 -- 172.21.15.79:0/4134198109 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c159d00 con 0x7f1f3c148f30 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.694+0000 7f1f4106c640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f3c148f30 0x7f1f3c1627b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009dd0 tx=0x7f1f3400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.695+0000 7f1f397fa640 1 -- 172.21.15.79:0/4134198109 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34018020 con 0x7f1f3c148f30 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.695+0000 7f1f397fa640 1 -- 172.21.15.79:0/4134198109 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f34014480 con 0x7f1f3c148f30 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.695+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c159fe0 con 0x7f1f3c148f30 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.695+0000 7f1f397fa640 1 -- 172.21.15.79:0/4134198109 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34014870 con 0x7f1f3c148f30 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.695+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c15a5a0 con 0x7f1f3c148f30 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.696+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3c0506c0 con 0x7f1f3c148f30 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.696+0000 7f1f397fa640 1 -- 172.21.15.79:0/4134198109 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f34005d10 con 0x7f1f3c148f30 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.696+0000 7f1f397fa640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1f1c06c8f0 0x7f1f1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.696+0000 7f1f397fa640 1 -- 172.21.15.79:0/4134198109 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1f3408c9f0 con 0x7f1f3c148f30 2024-04-26T19:51:27.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.696+0000 7f1f4086b640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1f1c06c8f0 0x7f1f1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.697+0000 7f1f4086b640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1f1c06c8f0 0x7f1f1c06edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000f60 tx=0x7f1f24007ce0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.701 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.701+0000 7f1f397fa640 1 -- 172.21.15.79:0/4134198109 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1f34022070 con 0x7f1f3c148f30 2024-04-26T19:51:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.877+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1f3c074220 con 0x7f1f3c148f30 2024-04-26T19:51:27.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.878+0000 7f1f397fa640 1 -- 172.21.15.79:0/4134198109 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f1f34010040 con 0x7f1f3c148f30 2024-04-26T19:51:27.878 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1f1c06c8f0 msgr2=0x7f1f1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1f1c06c8f0 0x7f1f1c06edb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1f24000f60 tx=0x7f1f24007ce0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f3c148f30 msgr2=0x7f1f3c1627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f3c148f30 0x7f1f3c1627b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009dd0 tx=0x7f1f3400c490 comp rx=0 tx=0).stop 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 shutdown_connections 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1f1c06c8f0 0x7f1f1c06edb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1f3c148f30 0x7f1f3c1627b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1f3c0b6630 0x7f1f3c13fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 --2- 172.21.15.79:0/4134198109 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1f3c06c350 0x7f1f3c1f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 >> 172.21.15.79:0/4134198109 conn(0x7f1f3c073b40 msgr2=0x7f1f3c0b6bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:27.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 shutdown_connections 2024-04-26T19:51:27.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.880+0000 7f1f42acf640 1 -- 172.21.15.79:0/4134198109 wait complete. 2024-04-26T19:51:27.892 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:27.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.990+0000 7efdffcc8640 1 Processor -- start 2024-04-26T19:51:27.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.990+0000 7efdffcc8640 1 -- start start 2024-04-26T19:51:27.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.991+0000 7efdffcc8640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efdf80d1830 0x7efdf81615e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:27.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.991+0000 7efdffcc8640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf80d1dd0 con 0x7efdf80b8f60 2024-04-26T19:51:27.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.991+0000 7efdffcc8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efdf80d20b0 con 0x7efdf806ca60 2024-04-26T19:51:27.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.991+0000 7efdffcc8640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7efdf80d2950 con 0x7efdf80d1830 2024-04-26T19:51:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.991+0000 7efdfda64640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7efdf80b8f60 0x7efdf806c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41458/0 (socket says 172.21.15.79:41458) 2024-04-26T19:51:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.991+0000 7efdfda64640 1 -- 172.21.15.79:0/1496062819 learned_addr learned my addr 172.21.15.79:0/1496062819 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.991+0000 7efdfe265640 1 --2- 172.21.15.79:0/1496062819 >> v2:172.21.15.79:3300/0 conn(0x7efdf80d1830 0x7efdf81615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.992+0000 7efdfca62640 1 -- 172.21.15.79:0/1496062819 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776089365 0 0) 0x7efdf80d1dd0 con 0x7efdf80b8f60 2024-04-26T19:51:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.992+0000 7efdfca62640 1 -- 172.21.15.79:0/1496062819 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efde4003520 con 0x7efdf80b8f60 2024-04-26T19:51:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.992+0000 7efdfe265640 1 -- 172.21.15.79:0/1496062819 >> v1:172.21.15.83:6789/0 conn(0x7efdf806ca60 legacy=0x7efdf8160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.992+0000 7efdfe265640 1 -- 172.21.15.79:0/1496062819 >> v1:172.21.15.79:6789/0 conn(0x7efdf80b8f60 legacy=0x7efdf806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.992+0000 7efdfe265640 1 -- 172.21.15.79:0/1496062819 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf80d2390 con 0x7efdf80d1830 2024-04-26T19:51:27.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.992+0000 7efdfe265640 1 --2- 172.21.15.79:0/1496062819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 0x7efdf81615e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efdf00097a0 tx=0x7efdf002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60540a31d983f1dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:27.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.992+0000 7efdfca62640 1 -- 172.21.15.79:0/1496062819 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0037070 con 0x7efdf80d1830 2024-04-26T19:51:27.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.993+0000 7efdfca62640 1 -- 172.21.15.79:0/1496062819 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf0032640 con 0x7efdf80d1830 2024-04-26T19:51:27.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.993+0000 7efdfca62640 1 -- 172.21.15.79:0/1496062819 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0032a10 con 0x7efdf80d1830 2024-04-26T19:51:27.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.993+0000 7efdffcc8640 1 -- 172.21.15.79:0/1496062819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 msgr2=0x7efdf81615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:27.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.993+0000 7efdffcc8640 1 --2- 172.21.15.79:0/1496062819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 0x7efdf81615e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7efdf00097a0 tx=0x7efdf002a2c0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.993+0000 7efdffcc8640 1 -- 172.21.15.79:0/1496062819 shutdown_connections 2024-04-26T19:51:27.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.993+0000 7efdffcc8640 1 --2- 172.21.15.79:0/1496062819 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 0x7efdf81615e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:27.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.993+0000 7efdffcc8640 1 -- 172.21.15.79:0/1496062819 >> 172.21.15.79:0/1496062819 conn(0x7efdf8073b40 msgr2=0x7efdf8073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:27.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.993+0000 7efdffcc8640 1 -- 172.21.15.79:0/1496062819 shutdown_connections 2024-04-26T19:51:27.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.993+0000 7efdffcc8640 1 -- 172.21.15.79:0/1496062819 wait complete. 2024-04-26T19:51:28.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.994+0000 7efdffcc8640 1 Processor -- start 2024-04-26T19:51:28.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.994+0000 7efdffcc8640 1 -- start start 2024-04-26T19:51:28.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdffcc8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdf806ca60 0x7efdf8150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdffcc8640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdf80b8f60 0x7efdf815b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdffcc8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 0x7efdf80b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdffcc8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efdf806d190 con 0x7efdf80d1830 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdffcc8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efdf806d010 con 0x7efdf806ca60 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdffcc8640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7efdf80b9340 con 0x7efdf80b8f60 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfe265640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 0x7efdf80b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfe265640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 0x7efdf80b7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34922/0 (socket says 172.21.15.79:34922) 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfe265640 1 -- 172.21.15.79:0/2378743396 learned_addr learned my addr 172.21.15.79:0/2378743396 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfda64640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdf806ca60 0x7efdf8150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfd263640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdf80b8f60 0x7efdf815b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfe265640 1 -- 172.21.15.79:0/2378743396 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdf80b8f60 msgr2=0x7efdf815b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfe265640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdf80b8f60 0x7efdf815b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfe265640 1 -- 172.21.15.79:0/2378743396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdf806ca60 msgr2=0x7efdf8150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfe265640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdf806ca60 0x7efdf8150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfe265640 1 -- 172.21.15.79:0/2378743396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf815a0d0 con 0x7efdf80d1830 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.995+0000 7efdfda64640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdf806ca60 0x7efdf8150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.996+0000 7efdfe265640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 0x7efdf80b7a70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efdf0035040 tx=0x7efdf00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.996+0000 7efdee7fc640 1 -- 172.21.15.79:0/2378743396 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0037040 con 0x7efdf80d1830 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.996+0000 7efdee7fc640 1 -- 172.21.15.79:0/2378743396 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdf0042070 con 0x7efdf80d1830 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.996+0000 7efdee7fc640 1 -- 172.21.15.79:0/2378743396 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0030910 con 0x7efdf80d1830 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.996+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf815a3b0 con 0x7efdf80d1830 2024-04-26T19:51:28.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.996+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf815a970 con 0x7efdf80d1830 2024-04-26T19:51:28.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.997+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf8068ce0 con 0x7efdf80d1830 2024-04-26T19:51:28.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.997+0000 7efdee7fc640 1 -- 172.21.15.79:0/2378743396 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efdf0030ab0 con 0x7efdf80d1830 2024-04-26T19:51:28.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.998+0000 7efdee7fc640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efddc06c8f0 0x7efddc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.998+0000 7efdfda64640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efddc06c8f0 0x7efddc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.998+0000 7efdee7fc640 1 -- 172.21.15.79:0/2378743396 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efdf00ac800 con 0x7efdf80d1830 2024-04-26T19:51:28.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:27.998+0000 7efdfda64640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efddc06c8f0 0x7efddc06edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efdf4001560 tx=0x7efdf4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:28.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.002+0000 7efdee7fc640 1 -- 172.21.15.79:0/2378743396 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efdf8068ce0 con 0x7efdf80d1830 2024-04-26T19:51:28.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.180+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efdf8074200 con 0x7efdf80d1830 2024-04-26T19:51:28.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.180+0000 7efdee7fc640 1 -- 172.21.15.79:0/2378743396 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7efdf0045360 con 0x7efdf80d1830 2024-04-26T19:51:28.181 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efddc06c8f0 msgr2=0x7efddc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efddc06c8f0 0x7efddc06edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efdf4001560 tx=0x7efdf4009040 comp rx=0 tx=0).stop 2024-04-26T19:51:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 msgr2=0x7efdf80b7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 0x7efdf80b7a70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7efdf0035040 tx=0x7efdf00046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 shutdown_connections 2024-04-26T19:51:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efddc06c8f0 0x7efddc06edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efdf80d1830 0x7efdf80b7a70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efdf80b8f60 0x7efdf815b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 --2- 172.21.15.79:0/2378743396 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efdf806ca60 0x7efdf8150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 >> 172.21.15.79:0/2378743396 conn(0x7efdf8073b40 msgr2=0x7efdf806fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:28.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 shutdown_connections 2024-04-26T19:51:28.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.182+0000 7efdffcc8640 1 -- 172.21.15.79:0/2378743396 wait complete. 2024-04-26T19:51:28.193 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:28.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.293+0000 7fa306874640 1 Processor -- start 2024-04-26T19:51:28.294 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.293+0000 7fa306874640 1 -- start start 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.294+0000 7fa306874640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa3001483d0 0x7fa300163270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.294+0000 7fa306874640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa30006ec90 0x7fa30006f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.294+0000 7fa306874640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3000b00a0 con 0x7fa3001637b0 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.294+0000 7fa306874640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fa3000701a0 con 0x7fa3001483d0 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.294+0000 7fa306874640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa3000af2e0 con 0x7fa30006ec90 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.294+0000 7fa2fffff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa3001483d0 0x7fa300163270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.294+0000 7fa2ff7fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa3001637b0 0x7fa300163c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41462/0 (socket says 172.21.15.79:41462) 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.294+0000 7fa2ff7fe640 1 -- 172.21.15.79:0/1247600986 learned_addr learned my addr 172.21.15.79:0/1247600986 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.294+0000 7fa304e11640 1 --2- 172.21.15.79:0/1247600986 >> v2:172.21.15.83:3300/0 conn(0x7fa30006ec90 0x7fa30006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.295+0000 7fa2fffff640 1 -- 172.21.15.79:0/1247600986 >> v2:172.21.15.83:3300/0 conn(0x7fa30006ec90 msgr2=0x7fa30006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:28.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.295+0000 7fa2fffff640 1 --2- 172.21.15.79:0/1247600986 >> v2:172.21.15.83:3300/0 conn(0x7fa30006ec90 0x7fa30006f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.295+0000 7fa2fffff640 1 -- 172.21.15.79:0/1247600986 >> v1:172.21.15.79:6789/0 conn(0x7fa3001637b0 legacy=0x7fa300163c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.295+0000 7fa2fffff640 1 -- 172.21.15.79:0/1247600986 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3000af5c0 con 0x7fa3001483d0 2024-04-26T19:51:28.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.295+0000 7fa2feffd640 1 -- 172.21.15.79:0/1247600986 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3768662475 0 0) 0x7fa3000b00a0 con 0x7fa3001637b0 2024-04-26T19:51:28.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.295+0000 7fa2fffff640 1 --2- 172.21.15.79:0/1247600986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 0x7fa300163270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa3000b0a70 tx=0x7fa2f002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a351a93dffc0c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:28.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.295+0000 7fa2feffd640 1 -- 172.21.15.79:0/1247600986 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0037070 con 0x7fa3001483d0 2024-04-26T19:51:28.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.295+0000 7fa2feffd640 1 -- 172.21.15.79:0/1247600986 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f0032440 con 0x7fa3001483d0 2024-04-26T19:51:28.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.295+0000 7fa2feffd640 1 -- 172.21.15.79:0/1247600986 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0032810 con 0x7fa3001483d0 2024-04-26T19:51:28.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.296+0000 7fa306874640 1 -- 172.21.15.79:0/1247600986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 msgr2=0x7fa300163270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.296+0000 7fa306874640 1 --2- 172.21.15.79:0/1247600986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 0x7fa300163270 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa3000b0a70 tx=0x7fa2f002a1a0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.299 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.296+0000 7fa306874640 1 -- 172.21.15.79:0/1247600986 shutdown_connections 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.296+0000 7fa306874640 1 --2- 172.21.15.79:0/1247600986 >> v2:172.21.15.83:3300/0 conn(0x7fa30006ec90 0x7fa30006f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.296+0000 7fa306874640 1 --2- 172.21.15.79:0/1247600986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 0x7fa300163270 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.296+0000 7fa306874640 1 -- 172.21.15.79:0/1247600986 >> 172.21.15.79:0/1247600986 conn(0x7fa3000685d0 msgr2=0x7fa300071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.296+0000 7fa306874640 1 -- 172.21.15.79:0/1247600986 shutdown_connections 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.296+0000 7fa306874640 1 -- 172.21.15.79:0/1247600986 wait complete. 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.297+0000 7fa306874640 1 Processor -- start 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.297+0000 7fa306874640 1 -- start start 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa306874640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30006ec90 0x7fa300142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa306874640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 0x7fa3001606e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa306874640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3001637b0 0x7fa300159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa306874640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa300149280 con 0x7fa3001483d0 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa306874640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa3000afb80 con 0x7fa30006ec90 2024-04-26T19:51:28.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa306874640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa3001487b0 con 0x7fa3001637b0 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 0x7fa3001606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2ff7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 0x7fa3001606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34932/0 (socket says 172.21.15.79:34932) 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2ff7fe640 1 -- 172.21.15.79:0/1089450637 learned_addr learned my addr 172.21.15.79:0/1089450637 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa304e11640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3001637b0 0x7fa300159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2fffff640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30006ec90 0x7fa300142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2ff7fe640 1 -- 172.21.15.79:0/1089450637 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3001637b0 msgr2=0x7fa300159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2ff7fe640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3001637b0 0x7fa300159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2ff7fe640 1 -- 172.21.15.79:0/1089450637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30006ec90 msgr2=0x7fa300142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2ff7fe640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30006ec90 0x7fa300142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2ff7fe640 1 -- 172.21.15.79:0/1089450637 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa300160c20 con 0x7fa3001483d0 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.298+0000 7fa2fffff640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30006ec90 0x7fa300142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:28.301 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.299+0000 7fa2ff7fe640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 0x7fa3001606e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c430 tx=0x7fa2f400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:28.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.300+0000 7fa2fcff9640 1 -- 172.21.15.79:0/1089450637 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f40074e0 con 0x7fa3001483d0 2024-04-26T19:51:28.303 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.300+0000 7fa2fcff9640 1 -- 172.21.15.79:0/1089450637 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2f40047e0 con 0x7fa3001483d0 2024-04-26T19:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.300+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30015c7d0 con 0x7fa3001483d0 2024-04-26T19:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.300+0000 7fa2fcff9640 1 -- 172.21.15.79:0/1089450637 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4004bd0 con 0x7fa3001483d0 2024-04-26T19:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.300+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30015ae40 con 0x7fa3001483d0 2024-04-26T19:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.301+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3000506c0 con 0x7fa3001483d0 2024-04-26T19:51:28.304 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.301+0000 7fa2fcff9640 1 -- 172.21.15.79:0/1089450637 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa2f4011070 con 0x7fa3001483d0 2024-04-26T19:51:28.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.302+0000 7fa2fcff9640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2ec06c8f0 0x7fa2ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.302+0000 7fa2fcff9640 1 -- 172.21.15.79:0/1089450637 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa2f408bad0 con 0x7fa3001483d0 2024-04-26T19:51:28.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.302+0000 7fa2fffff640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2ec06c8f0 0x7fa2ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.302+0000 7fa2fffff640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2ec06c8f0 0x7fa2ec06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0034000 tx=0x7fa2f002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:28.307 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.306+0000 7fa2fcff9640 1 -- 172.21.15.79:0/1089450637 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa300160c20 con 0x7fa3001483d0 2024-04-26T19:51:28.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.482+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa30006a800 con 0x7fa3001483d0 2024-04-26T19:51:28.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.482+0000 7fa2fcff9640 1 -- 172.21.15.79:0/1089450637 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fa2f4059b00 con 0x7fa3001483d0 2024-04-26T19:51:28.483 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2ec06c8f0 msgr2=0x7fa2ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2ec06c8f0 0x7fa2ec06edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0034000 tx=0x7fa2f002e000 comp rx=0 tx=0).stop 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 msgr2=0x7fa3001606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 0x7fa3001606e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa2f400c430 tx=0x7fa2f400c8f0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 shutdown_connections 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa2ec06c8f0 0x7fa2ec06edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa3001637b0 0x7fa300159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa3001483d0 0x7fa3001606e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 --2- 172.21.15.79:0/1089450637 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa30006ec90 0x7fa300142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.485 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 >> 172.21.15.79:0/1089450637 conn(0x7fa3000685d0 msgr2=0x7fa3000d4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:28.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 shutdown_connections 2024-04-26T19:51:28.486 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.484+0000 7fa306874640 1 -- 172.21.15.79:0/1089450637 wait complete. 2024-04-26T19:51:28.496 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T19:51:28.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.593+0000 7f197f825640 1 Processor -- start 2024-04-26T19:51:28.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.593+0000 7f197f825640 1 -- start start 2024-04-26T19:51:28.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197f825640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f19780700d0 0x7f19780d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197f825640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f19780b64f0 0x7f19780b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197f825640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1978070740 con 0x7f197815f2e0 2024-04-26T19:51:28.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197f825640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f1978070510 con 0x7f19780b64f0 2024-04-26T19:51:28.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197f825640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f19780cec20 con 0x7f19780700d0 2024-04-26T19:51:28.595 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197ddc2640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f19780b64f0 0x7f19780b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197ddc2640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f19780b64f0 0x7f19780b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34934/0 (socket says 172.21.15.79:34934) 2024-04-26T19:51:28.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197ddc2640 1 -- 172.21.15.79:0/2440983551 learned_addr learned my addr 172.21.15.79:0/2440983551 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:28.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197ddc2640 1 -- 172.21.15.79:0/2440983551 >> v2:172.21.15.83:3300/0 conn(0x7f19780700d0 msgr2=0x7f19780d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:28.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197cdc0640 1 --2- 172.21.15.79:0/2440983551 >> v2:172.21.15.83:3300/0 conn(0x7f19780700d0 0x7f19780d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197ddc2640 1 --2- 172.21.15.79:0/2440983551 >> v2:172.21.15.83:3300/0 conn(0x7f19780700d0 0x7f19780d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197ddc2640 1 -- 172.21.15.79:0/2440983551 >> v1:172.21.15.83:6789/0 conn(0x7f197815f2e0 legacy=0x7f19780cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:28.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197ddc2640 1 -- 172.21.15.79:0/2440983551 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19780ce5d0 con 0x7f19780b64f0 2024-04-26T19:51:28.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.594+0000 7f197cdc0640 1 --2- 172.21.15.79:0/2440983551 >> v2:172.21.15.83:3300/0 conn(0x7f19780700d0 0x7f19780d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:28.596 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.595+0000 7f197ddc2640 1 --2- 172.21.15.79:0/2440983551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780b64f0 0x7f19780b6920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f19780b70b0 tx=0x7f197002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7078077aee87d62a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:28.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.595+0000 7f196ffff640 1 -- 172.21.15.79:0/2440983551 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1970036070 con 0x7f19780b64f0 2024-04-26T19:51:28.599 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.595+0000 7f196ffff640 1 -- 172.21.15.79:0/2440983551 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1970031600 con 0x7f19780b64f0 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.595+0000 7f196ffff640 1 -- 172.21.15.79:0/2440983551 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700319d0 con 0x7f19780b64f0 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.596+0000 7f197f825640 1 -- 172.21.15.79:0/2440983551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780b64f0 msgr2=0x7f19780b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.596+0000 7f197f825640 1 --2- 172.21.15.79:0/2440983551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780b64f0 0x7f19780b6920 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f19780b70b0 tx=0x7f197002a6f0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.596+0000 7f197f825640 1 -- 172.21.15.79:0/2440983551 shutdown_connections 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.596+0000 7f197f825640 1 --2- 172.21.15.79:0/2440983551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780b64f0 0x7f19780b6920 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.596+0000 7f197f825640 1 --2- 172.21.15.79:0/2440983551 >> v2:172.21.15.83:3300/0 conn(0x7f19780700d0 0x7f19780d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.596+0000 7f197f825640 1 -- 172.21.15.79:0/2440983551 >> 172.21.15.79:0/2440983551 conn(0x7f197806c770 msgr2=0x7f197806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.596+0000 7f197f825640 1 -- 172.21.15.79:0/2440983551 shutdown_connections 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.596+0000 7f197f825640 1 -- 172.21.15.79:0/2440983551 wait complete. 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.597+0000 7f197f825640 1 Processor -- start 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.597+0000 7f197f825640 1 -- start start 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197f825640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780700d0 0x7f1978160d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197f825640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19780b64f0 0x7f1978149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197f825640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f197815f2e0 0x7f19781584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.600 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197f825640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f19780d0ab0 con 0x7f19780700d0 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197f825640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f19780b6e60 con 0x7f19780b64f0 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197f825640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1978070a20 con 0x7f197815f2e0 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197d5c1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780700d0 0x7f1978160d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197d5c1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780700d0 0x7f1978160d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34946/0 (socket says 172.21.15.79:34946) 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197d5c1640 1 -- 172.21.15.79:0/2083226856 learned_addr learned my addr 172.21.15.79:0/2083226856 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197d5c1640 1 -- 172.21.15.79:0/2083226856 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f197815f2e0 msgr2=0x7f19781584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197d5c1640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f197815f2e0 0x7f19781584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197d5c1640 1 -- 172.21.15.79:0/2083226856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19780b64f0 msgr2=0x7f1978149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197d5c1640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19780b64f0 0x7f1978149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197d5c1640 1 -- 172.21.15.79:0/2083226856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1978161ff0 con 0x7f19780700d0 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197d5c1640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780700d0 0x7f1978160d70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f197400b420 tx=0x7f197400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f196dffb640 1 -- 172.21.15.79:0/2083226856 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974011830 con 0x7f19780700d0 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f196dffb640 1 -- 172.21.15.79:0/2083226856 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1974016070 con 0x7f19780700d0 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f196dffb640 1 -- 172.21.15.79:0/2083226856 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974011c10 con 0x7f19780700d0 2024-04-26T19:51:28.601 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1978160290 con 0x7f19780700d0 2024-04-26T19:51:28.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.598+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1978161c10 con 0x7f19780700d0 2024-04-26T19:51:28.602 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.599+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19780506c0 con 0x7f19780700d0 2024-04-26T19:51:28.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.600+0000 7f196dffb640 1 -- 172.21.15.79:0/2083226856 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f197401b460 con 0x7f19780700d0 2024-04-26T19:51:28.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.600+0000 7f196dffb640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f195c06c920 0x7f195c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.600+0000 7f196dffb640 1 -- 172.21.15.79:0/2083226856 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f197408ba00 con 0x7f19780700d0 2024-04-26T19:51:28.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.600+0000 7f197cdc0640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f195c06c920 0x7f195c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.603 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.601+0000 7f197cdc0640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f195c06c920 0x7f195c06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1968006fd0 tx=0x7f1968008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:28.605 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.604+0000 7f196dffb640 1 -- 172.21.15.79:0/2083226856 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1974020020 con 0x7f19780700d0 2024-04-26T19:51:28.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.790+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f197815c520 con 0x7f19780700d0 2024-04-26T19:51:28.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.791+0000 7f196dffb640 1 -- 172.21.15.79:0/2083226856 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f1974059a30 con 0x7f19780700d0 2024-04-26T19:51:28.791 INFO:teuthology.orchestra.run.smithi079.stderr:listed 0 entries 2024-04-26T19:51:28.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f195c06c920 msgr2=0x7f195c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f195c06c920 0x7f195c06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1968006fd0 tx=0x7f1968008040 comp rx=0 tx=0).stop 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780700d0 msgr2=0x7f1978160d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780700d0 0x7f1978160d70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f197400b420 tx=0x7f197400b8e0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 shutdown_connections 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f195c06c920 0x7f195c06ede0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f197815f2e0 0x7f19781584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f19780b64f0 0x7f1978149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 --2- 172.21.15.79:0/2083226856 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f19780700d0 0x7f1978160d70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 >> 172.21.15.79:0/2083226856 conn(0x7f197806c770 msgr2=0x7f19780b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 shutdown_connections 2024-04-26T19:51:28.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.793+0000 7f197f825640 1 -- 172.21.15.79:0/2083226856 wait complete. 2024-04-26T19:51:28.804 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-26T19:51:28.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:28.805 DEBUG:teuthology.orchestra.run.smithi083:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-26T19:51:28.834 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:51:28.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:28.835 DEBUG:teuthology.orchestra.run.smithi083:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-26T19:51:28.893 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-26T19:51:28.893 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:51:28.990 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.990+0000 7f90bd6fc640 1 Processor -- start 2024-04-26T19:51:28.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.990+0000 7f90bd6fc640 1 -- start start 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90bd6fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f90b8148fd0 0x7f90b80d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90bd6fc640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b80b1290 con 0x7f90b806ff00 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90bd6fc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b80d4090 con 0x7f90b806fb20 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90bd6fc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f90b80d4370 con 0x7f90b8148fd0 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b67fc640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f90b806ff00 0x7f90b80aebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41478/0 (socket says 172.21.15.79:41478) 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b67fc640 1 -- 172.21.15.79:0/3011438839 learned_addr learned my addr 172.21.15.79:0/3011438839 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3345900557 0 0) 0x7f90b80b1290 con 0x7f90b806ff00 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90a0003520 con 0x7f90b806ff00 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b77fe640 1 --2- 172.21.15.79:0/3011438839 >> v2:172.21.15.83:3300/0 conn(0x7f90b8148fd0 0x7f90b80d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 895062362 0 0) 0x7f90a0003520 con 0x7f90b806ff00 2024-04-26T19:51:28.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 >> v2:172.21.15.83:3300/0 conn(0x7f90b8148fd0 msgr2=0x7f90b80d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:28.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 --2- 172.21.15.79:0/3011438839 >> v2:172.21.15.83:3300/0 conn(0x7f90b8148fd0 0x7f90b80d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 >> v1:172.21.15.83:6789/0 conn(0x7f90b806fb20 legacy=0x7f90b81488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b80d4650 con 0x7f90b806ff00 2024-04-26T19:51:28.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f90ac001670 con 0x7f90b806ff00 2024-04-26T19:51:28.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 >> v1:172.21.15.79:6789/0 conn(0x7f90b806ff00 legacy=0x7f90b80aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a00039e0 0x7f90a0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f90a00051c0 0x7f90a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:28.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90b80d4370 con 0x7f90a0004800 2024-04-26T19:51:28.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90a000c070 con 0x7f90a00039e0 2024-04-26T19:51:28.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f90a000c270 con 0x7f90a00051c0 2024-04-26T19:51:28.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b67fc640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.991+0000 7f90b77fe640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f90a00051c0 0x7f90a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b6ffd640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a00039e0 0x7f90a0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b67fc640 1 -- 172.21.15.79:0/3011438839 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f90a00051c0 msgr2=0x7f90a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b67fc640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f90a00051c0 0x7f90a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b67fc640 1 -- 172.21.15.79:0/3011438839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a00039e0 msgr2=0x7f90a0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b67fc640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90a00039e0 0x7f90a0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b67fc640 1 -- 172.21.15.79:0/3011438839 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a0003520 con 0x7f90a0004800 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b77fe640 1 -- 172.21.15.79:0/3011438839 reap_dead start 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b67fc640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90a0004c80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00c240 tx=0x7f90a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f05c5088b7f017d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac007cf0 con 0x7f90a0004800 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90ac005af0 con 0x7f90a0004800 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.992+0000 7f90b5ffb640 1 -- 172.21.15.79:0/3011438839 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90ac007070 con 0x7f90a0004800 2024-04-26T19:51:28.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.993+0000 7f90bd6fc640 1 -- 172.21.15.79:0/3011438839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 msgr2=0x7f90a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:28.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.993+0000 7f90bd6fc640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90a0004c80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00c240 tx=0x7f90a0005830 comp rx=0 tx=0).stop 2024-04-26T19:51:28.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.993+0000 7f90bd6fc640 1 -- 172.21.15.79:0/3011438839 shutdown_connections 2024-04-26T19:51:28.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.993+0000 7f90bd6fc640 1 --2- 172.21.15.79:0/3011438839 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90a0004c80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:28.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.993+0000 7f90bd6fc640 1 -- 172.21.15.79:0/3011438839 >> 172.21.15.79:0/3011438839 conn(0x7f90b8071c30 msgr2=0x7f90b806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:28.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.993+0000 7f90bd6fc640 1 -- 172.21.15.79:0/3011438839 shutdown_connections 2024-04-26T19:51:28.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.993+0000 7f90bd6fc640 1 -- 172.21.15.79:0/3011438839 wait complete. 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.994+0000 7f90bd6fc640 1 Processor -- start 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.994+0000 7f90bd6fc640 1 -- start start 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90bd6fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90b80d1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90bd6fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90b8160bb0 0x7f90b8161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90bd6fc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f90b81f8550 0x7f90b81f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90bd6fc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f90b80d2cd0 con 0x7f90a0004800 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90bd6fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f90b80d4c10 con 0x7f90b8160bb0 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90bd6fc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f90b80d2e50 con 0x7f90b81f8550 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90b80d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b6ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90b80d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34968/0 (socket says 172.21.15.79:34968) 2024-04-26T19:51:29.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b6ffd640 1 -- 172.21.15.79:0/361613100 learned_addr learned my addr 172.21.15.79:0/361613100 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b77fe640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f90b81f8550 0x7f90b81f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b67fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90b8160bb0 0x7f90b8161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b6ffd640 1 -- 172.21.15.79:0/361613100 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f90b81f8550 msgr2=0x7f90b81f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b6ffd640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f90b81f8550 0x7f90b81f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b6ffd640 1 -- 172.21.15.79:0/361613100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90b8160bb0 msgr2=0x7f90b8161010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b6ffd640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90b8160bb0 0x7f90b8161010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b6ffd640 1 -- 172.21.15.79:0/361613100 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b81415b0 con 0x7f90a0004800 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b67fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90b8160bb0 0x7f90b8161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.995+0000 7f90b6ffd640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90b80d1280 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f90a800e410 tx=0x7f90a800e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:29.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.996+0000 7f909f7fe640 1 -- 172.21.15.79:0/361613100 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8019070 con 0x7f90a0004800 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.996+0000 7f909f7fe640 1 -- 172.21.15.79:0/361613100 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a80052e0 con 0x7f90a0004800 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.996+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b8144790 con 0x7f90a0004800 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.996+0000 7f909f7fe640 1 -- 172.21.15.79:0/361613100 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8003810 con 0x7f90a0004800 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.996+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b8144d30 con 0x7f90a0004800 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.997+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90b80506c0 con 0x7f90a0004800 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.997+0000 7f909f7fe640 1 -- 172.21.15.79:0/361613100 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90a801c070 con 0x7f90a0004800 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.997+0000 7f909f7fe640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f90a406c920 0x7f90a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.998+0000 7f909f7fe640 1 -- 172.21.15.79:0/361613100 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f90a808b4c0 con 0x7f90a0004800 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.998+0000 7f90b67fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f90a406c920 0x7f90a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:28.998+0000 7f90b67fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f90a406c920 0x7f90a406ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90ac032000 tx=0x7f90ac009030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:29.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.002+0000 7f909f7fe640 1 -- 172.21.15.79:0/361613100 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f90a8058ea0 con 0x7f90a0004800 2024-04-26T19:51:29.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.212+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90b80d3700 con 0x7f90a0004800 2024-04-26T19:51:29.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.212+0000 7f909f7fe640 1 -- 172.21.15.79:0/361613100 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f90a805d1a0 con 0x7f90a0004800 2024-04-26T19:51:29.213 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:29.213 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:52.517190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:51:29.214 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2024-04-26T19:51:29.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.214+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f90a406c920 msgr2=0x7f90a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.214+0000 7f90bd6fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f90a406c920 0x7f90a406ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f90ac032000 tx=0x7f90ac009030 comp rx=0 tx=0).stop 2024-04-26T19:51:29.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.214+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 msgr2=0x7f90b80d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.214+0000 7f90bd6fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90b80d1280 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f90a800e410 tx=0x7f90a800e8d0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.215+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 shutdown_connections 2024-04-26T19:51:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.215+0000 7f90bd6fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f90a406c920 0x7f90a406ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.215+0000 7f90bd6fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f90b81f8550 0x7f90b81f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.215+0000 7f90bd6fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f90b8160bb0 0x7f90b8161010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.215+0000 7f90bd6fc640 1 --2- 172.21.15.79:0/361613100 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f90a0004800 0x7f90b80d1280 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.215+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 >> 172.21.15.79:0/361613100 conn(0x7f90b8071c30 msgr2=0x7f90b8072010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.215+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 shutdown_connections 2024-04-26T19:51:29.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.215+0000 7f90bd6fc640 1 -- 172.21.15.79:0/361613100 wait complete. 2024-04-26T19:51:29.227 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.323+0000 7f018cdda640 1 Processor -- start 2024-04-26T19:51:29.324 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.323+0000 7f018cdda640 1 -- start start 2024-04-26T19:51:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.324+0000 7f018cdda640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 0x7f01880d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.324+0000 7f018cdda640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f01880b4160 0x7f01880aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.324+0000 7f018cdda640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01880d15d0 con 0x7f01880d25b0 2024-04-26T19:51:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.324+0000 7f018cdda640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f01880d2f80 con 0x7f01880b4160 2024-04-26T19:51:29.325 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.324+0000 7f018cdda640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0188161a20 con 0x7f01880b3d80 2024-04-26T19:51:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.324+0000 7f0185d9b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f01880b4160 0x7f01880aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.324+0000 7f0185d9b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f01880b4160 0x7f01880aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34980/0 (socket says 172.21.15.79:34980) 2024-04-26T19:51:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.324+0000 7f0185d9b640 1 -- 172.21.15.79:0/3437881482 learned_addr learned my addr 172.21.15.79:0/3437881482 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.324+0000 7f018659c640 1 --2- 172.21.15.79:0/3437881482 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 0x7f01880d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.325+0000 7f0185d9b640 1 -- 172.21.15.79:0/3437881482 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 msgr2=0x7f01880d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.325+0000 7f0185d9b640 1 --2- 172.21.15.79:0/3437881482 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 0x7f01880d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.325+0000 7f0185d9b640 1 -- 172.21.15.79:0/3437881482 >> v1:172.21.15.83:6789/0 conn(0x7f01880d25b0 legacy=0x7f0188160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.326 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.325+0000 7f0185d9b640 1 -- 172.21.15.79:0/3437881482 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0188161d00 con 0x7f01880b4160 2024-04-26T19:51:29.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.325+0000 7f0185d9b640 1 --2- 172.21.15.79:0/3437881482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 0x7f01880aa4c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f017c009870 tx=0x7f017c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f06f855c9188ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:29.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.325+0000 7f018559a640 1 -- 172.21.15.79:0/3437881482 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c037070 con 0x7f01880b4160 2024-04-26T19:51:29.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.325+0000 7f018559a640 1 -- 172.21.15.79:0/3437881482 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f017c032440 con 0x7f01880b4160 2024-04-26T19:51:29.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.325+0000 7f018559a640 1 -- 172.21.15.79:0/3437881482 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c032810 con 0x7f01880b4160 2024-04-26T19:51:29.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.326+0000 7f018cdda640 1 -- 172.21.15.79:0/3437881482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 msgr2=0x7f01880aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.326+0000 7f018cdda640 1 --2- 172.21.15.79:0/3437881482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 0x7f01880aa4c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f017c009870 tx=0x7f017c02a270 comp rx=0 tx=0).stop 2024-04-26T19:51:29.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.326+0000 7f018cdda640 1 -- 172.21.15.79:0/3437881482 shutdown_connections 2024-04-26T19:51:29.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.326+0000 7f018cdda640 1 --2- 172.21.15.79:0/3437881482 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 0x7f01880aa4c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.326+0000 7f018cdda640 1 --2- 172.21.15.79:0/3437881482 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 0x7f01880d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.326+0000 7f018cdda640 1 -- 172.21.15.79:0/3437881482 >> 172.21.15.79:0/3437881482 conn(0x7f01880b2720 msgr2=0x7f0188070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:29.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.326+0000 7f018cdda640 1 -- 172.21.15.79:0/3437881482 shutdown_connections 2024-04-26T19:51:29.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.326+0000 7f018cdda640 1 -- 172.21.15.79:0/3437881482 wait complete. 2024-04-26T19:51:29.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.327+0000 7f018cdda640 1 Processor -- start 2024-04-26T19:51:29.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.327+0000 7f018cdda640 1 -- start start 2024-04-26T19:51:29.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f018cdda640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 0x7f018813fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f018cdda640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 0x7f018815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f018cdda640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01880d25b0 0x7f018813a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f018cdda640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f01881615e0 con 0x7f01880b4160 2024-04-26T19:51:29.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f018cdda640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f01880d1b50 con 0x7f01880d25b0 2024-04-26T19:51:29.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f018cdda640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0188161760 con 0x7f01880b3d80 2024-04-26T19:51:29.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0185d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 0x7f018815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0185d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 0x7f018815c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34982/0 (socket says 172.21.15.79:34982) 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0185d9b640 1 -- 172.21.15.79:0/1734489180 learned_addr learned my addr 172.21.15.79:0/1734489180 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f018659c640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 0x7f018813fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0185d9b640 1 -- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 msgr2=0x7f018813fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0186d9d640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01880d25b0 0x7f018813a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0185d9b640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 0x7f018813fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0185d9b640 1 -- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01880d25b0 msgr2=0x7f018813a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0185d9b640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01880d25b0 0x7f018813a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0185d9b640 1 -- 172.21.15.79:0/1734489180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01881402b0 con 0x7f01880b4160 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f018659c640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 0x7f018813fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.328+0000 7f0186d9d640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01880d25b0 0x7f018813a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:29.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.329+0000 7f0185d9b640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 0x7f018815c600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f017c000c00 tx=0x7f017c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.329+0000 7f0176ffd640 1 -- 172.21.15.79:0/1734489180 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c041020 con 0x7f01880b4160 2024-04-26T19:51:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.329+0000 7f0176ffd640 1 -- 172.21.15.79:0/1734489180 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f017c0306c0 con 0x7f01880b4160 2024-04-26T19:51:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.329+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0188140590 con 0x7f01880b4160 2024-04-26T19:51:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.329+0000 7f0176ffd640 1 -- 172.21.15.79:0/1734489180 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017c037040 con 0x7f01880b4160 2024-04-26T19:51:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.329+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0188140b50 con 0x7f01880b4160 2024-04-26T19:51:29.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.330+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01880506c0 con 0x7f01880b4160 2024-04-26T19:51:29.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.330+0000 7f0176ffd640 1 -- 172.21.15.79:0/1734489180 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f017c044020 con 0x7f01880b4160 2024-04-26T19:51:29.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.331+0000 7f0176ffd640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f016806c8f0 0x7f016806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.331+0000 7f0176ffd640 1 -- 172.21.15.79:0/1734489180 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f017c0ac6a0 con 0x7f01880b4160 2024-04-26T19:51:29.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.331+0000 7f018659c640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f016806c8f0 0x7f016806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.331+0000 7f018659c640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f016806c8f0 0x7f016806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0178009f50 tx=0x7f017800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:29.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.335+0000 7f0176ffd640 1 -- 172.21.15.79:0/1734489180 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f017c04b030 con 0x7f01880b4160 2024-04-26T19:51:29.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.511+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01880d13b0 con 0x7f01880b4160 2024-04-26T19:51:29.513 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.512+0000 7f0176ffd640 1 -- 172.21.15.79:0/1734489180 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7f017c07a760 con 0x7f01880b4160 2024-04-26T19:51:29.513 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:29.513 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:29.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f016806c8f0 msgr2=0x7f016806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f016806c8f0 0x7f016806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0178009f50 tx=0x7f017800b040 comp rx=0 tx=0).stop 2024-04-26T19:51:29.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 msgr2=0x7f018815c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 0x7f018815c600 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f017c000c00 tx=0x7f017c0046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 shutdown_connections 2024-04-26T19:51:29.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f016806c8f0 0x7f016806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f01880d25b0 0x7f018813a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f01880b4160 0x7f018815c600 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 --2- 172.21.15.79:0/1734489180 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f01880b3d80 0x7f018813fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 >> 172.21.15.79:0/1734489180 conn(0x7f01880b2720 msgr2=0x7f018815f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:29.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 shutdown_connections 2024-04-26T19:51:29.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.514+0000 7f018cdda640 1 -- 172.21.15.79:0/1734489180 wait complete. 2024-04-26T19:51:29.527 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:50:28.469735+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:50:29.397999+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-26T19:50:33.118806+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-26T19:51:29.528 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:29.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.628+0000 7f9dd7ebd640 1 Processor -- start 2024-04-26T19:51:29.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.628+0000 7f9dd7ebd640 1 -- start start 2024-04-26T19:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd7ebd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd00b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd7ebd640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9dd00b7620 0x7f9dd00b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd7ebd640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dd00b7e60 con 0x7f9dd006c350 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd7ebd640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9dd00b8140 con 0x7f9dd00b7620 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd7ebd640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd00b8420 con 0x7f9dd0148f30 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd5c59640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd00b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd5458640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9dd00b7620 0x7f9dd00b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd5c59640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd00b6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:37572/0 (socket says 172.21.15.79:37572) 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd5458640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9dd00b7620 0x7f9dd00b8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:49436/0 (socket says 172.21.15.79:49436) 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd645a640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9dd006c350 0x7f9dd015fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:44132/0 (socket says 172.21.15.79:44132) 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.629+0000 7f9dd5c59640 1 -- 172.21.15.79:0/1253977580 learned_addr learned my addr 172.21.15.79:0/1253977580 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.630+0000 7f9dd4c57640 1 -- 172.21.15.79:0/1253977580 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 933518299 0 0) 0x7f9dd00b7e60 con 0x7f9dd006c350 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.630+0000 7f9dd4c57640 1 -- 172.21.15.79:0/1253977580 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9dbc003520 con 0x7f9dd006c350 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.630+0000 7f9dd5458640 1 -- 172.21.15.79:0/1253977580 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 msgr2=0x7f9dd00b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:29.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.630+0000 7f9dd5458640 1 --2- 172.21.15.79:0/1253977580 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd00b6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.630+0000 7f9dd5458640 1 -- 172.21.15.79:0/1253977580 >> v1:172.21.15.83:6789/0 conn(0x7f9dd006c350 legacy=0x7f9dd015fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.630+0000 7f9dd5458640 1 -- 172.21.15.79:0/1253977580 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd00b8700 con 0x7f9dd00b7620 2024-04-26T19:51:29.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.630+0000 7f9dd5c59640 1 --2- 172.21.15.79:0/1253977580 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd00b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:29.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd5458640 1 --2- 172.21.15.79:0/1253977580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd00b7620 0x7f9dd00b8f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00097a0 tx=0x7f9dc002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cf6cf092b4b1b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:29.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd4c57640 1 -- 172.21.15.79:0/1253977580 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0037070 con 0x7f9dd00b7620 2024-04-26T19:51:29.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd4c57640 1 -- 172.21.15.79:0/1253977580 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dc0032440 con 0x7f9dd00b7620 2024-04-26T19:51:29.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd4c57640 1 -- 172.21.15.79:0/1253977580 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00327f0 con 0x7f9dd00b7620 2024-04-26T19:51:29.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/1253977580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd00b7620 msgr2=0x7f9dd00b8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd7ebd640 1 --2- 172.21.15.79:0/1253977580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd00b7620 0x7f9dd00b8f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00097a0 tx=0x7f9dc002a180 comp rx=0 tx=0).stop 2024-04-26T19:51:29.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/1253977580 shutdown_connections 2024-04-26T19:51:29.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd7ebd640 1 --2- 172.21.15.79:0/1253977580 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd00b7620 0x7f9dd00b8f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd7ebd640 1 --2- 172.21.15.79:0/1253977580 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd00b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/1253977580 >> 172.21.15.79:0/1253977580 conn(0x7f9dd0073b40 msgr2=0x7f9dd0073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/1253977580 shutdown_connections 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.631+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/1253977580 wait complete. 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.632+0000 7f9dd7ebd640 1 Processor -- start 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.632+0000 7f9dd7ebd640 1 -- start start 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd7ebd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dd006c350 0x7f9dd013fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd7ebd640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd00b7620 0x7f9dd0160e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd7ebd640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd006f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd7ebd640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00b7a60 con 0x7f9dd006c350 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd7ebd640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9dd00b9440 con 0x7f9dd00b7620 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd7ebd640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9dd00b7be0 con 0x7f9dd0148f30 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd5c59640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dd006c350 0x7f9dd013fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.638 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd5c59640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dd006c350 0x7f9dd013fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34986/0 (socket says 172.21.15.79:34986) 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd5c59640 1 -- 172.21.15.79:0/154642020 learned_addr learned my addr 172.21.15.79:0/154642020 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd5458640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd00b7620 0x7f9dd0160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.633+0000 7f9dd645a640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd006f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dd5c59640 1 -- 172.21.15.79:0/154642020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 msgr2=0x7f9dd006f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dd5c59640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd006f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dd5c59640 1 -- 172.21.15.79:0/154642020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd00b7620 msgr2=0x7f9dd0160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dd5c59640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd00b7620 0x7f9dd0160e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dd5c59640 1 -- 172.21.15.79:0/154642020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dd01402b0 con 0x7f9dd006c350 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dd645a640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd006f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dd5c59640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dd006c350 0x7f9dd013fd70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00e9b0 tx=0x7f9dcc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dc67fc640 1 -- 172.21.15.79:0/154642020 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc017070 con 0x7f9dd006c350 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dc67fc640 1 -- 172.21.15.79:0/154642020 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dcc00c9a0 con 0x7f9dd006c350 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dd015fb50 con 0x7f9dd006c350 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dc67fc640 1 -- 172.21.15.79:0/154642020 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc00cd50 con 0x7f9dd006c350 2024-04-26T19:51:29.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.634+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dd0160390 con 0x7f9dd006c350 2024-04-26T19:51:29.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.635+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dd00506c0 con 0x7f9dd006c350 2024-04-26T19:51:29.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.635+0000 7f9dc67fc640 1 -- 172.21.15.79:0/154642020 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9dcc01a070 con 0x7f9dd006c350 2024-04-26T19:51:29.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.636+0000 7f9dc67fc640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9db406c920 0x7f9db406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:29.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.636+0000 7f9dd5458640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9db406c920 0x7f9db406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:29.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.636+0000 7f9dc67fc640 1 -- 172.21.15.79:0/154642020 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9dcc08be50 con 0x7f9dd006c350 2024-04-26T19:51:29.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.636+0000 7f9dd5458640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9db406c920 0x7f9db406ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0034000 tx=0x7f9dc002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:29.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.640+0000 7f9dc67fc640 1 -- 172.21.15.79:0/154642020 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9dd015fb50 con 0x7f9dd006c350 2024-04-26T19:51:29.815 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.815+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9dd00d24b0 con 0x7f9dd006c350 2024-04-26T19:51:29.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.815+0000 7f9dc67fc640 1 -- 172.21.15.79:0/154642020 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f9dcc059df0 con 0x7f9dd006c350 2024-04-26T19:51:29.816 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:29.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9db406c920 msgr2=0x7f9db406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9db406c920 0x7f9db406ede0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0034000 tx=0x7f9dc002e000 comp rx=0 tx=0).stop 2024-04-26T19:51:29.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dd006c350 msgr2=0x7f9dd013fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:29.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dd006c350 0x7f9dd013fd70 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc00e9b0 tx=0x7f9dcc00ee70 comp rx=0 tx=0).stop 2024-04-26T19:51:29.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 shutdown_connections 2024-04-26T19:51:29.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9db406c920 0x7f9db406ede0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9dd0148f30 0x7f9dd006f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9dd00b7620 0x7f9dd0160e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 --2- 172.21.15.79:0/154642020 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9dd006c350 0x7f9dd013fd70 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:29.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 >> 172.21.15.79:0/154642020 conn(0x7f9dd0073b40 msgr2=0x7f9dd00b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:29.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 shutdown_connections 2024-04-26T19:51:29.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:29.817+0000 7f9dd7ebd640 1 -- 172.21.15.79:0/154642020 wait complete. 2024-04-26T19:51:29.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:51:30.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.025+0000 7fc47e189640 1 Processor -- start 2024-04-26T19:51:30.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.025+0000 7fc47e189640 1 -- start start 2024-04-26T19:51:30.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc47e189640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 0x7fc478073bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc47e189640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc4780718f0 0x7fc478071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc47e189640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc478072620 con 0x7fc47815c1e0 2024-04-26T19:51:30.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc47e189640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fc4780730d0 con 0x7fc4780718f0 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc47e189640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc4780733b0 con 0x7fc478070030 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc477fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc4780718f0 0x7fc478071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc477fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc4780718f0 0x7fc478071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:34992/0 (socket says 172.21.15.79:34992) 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc477fff640 1 -- 172.21.15.79:0/3675775967 learned_addr learned my addr 172.21.15.79:0/3675775967 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc476ffd640 1 --2- 172.21.15.79:0/3675775967 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 0x7fc478073bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc477fff640 1 -- 172.21.15.79:0/3675775967 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 msgr2=0x7fc478073bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc477fff640 1 --2- 172.21.15.79:0/3675775967 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 0x7fc478073bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc477fff640 1 -- 172.21.15.79:0/3675775967 >> v1:172.21.15.83:6789/0 conn(0x7fc47815c1e0 legacy=0x7fc47806f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc477fff640 1 -- 172.21.15.79:0/3675775967 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc478073690 con 0x7fc4780718f0 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.026+0000 7fc476ffd640 1 --2- 172.21.15.79:0/3675775967 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 0x7fc478073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.027+0000 7fc477fff640 1 --2- 172.21.15.79:0/3675775967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 0x7fc478071d20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc47000d150 tx=0x7fc47002c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed47cdb055068c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.027+0000 7fc4767fc640 1 -- 172.21.15.79:0/3675775967 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470009600 con 0x7fc4780718f0 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.027+0000 7fc4767fc640 1 -- 172.21.15.79:0/3675775967 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4700097a0 con 0x7fc4780718f0 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.027+0000 7fc4767fc640 1 -- 172.21.15.79:0/3675775967 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470009b70 con 0x7fc4780718f0 2024-04-26T19:51:30.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.028+0000 7fc47e189640 1 -- 172.21.15.79:0/3675775967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 msgr2=0x7fc478071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.028+0000 7fc47e189640 1 --2- 172.21.15.79:0/3675775967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 0x7fc478071d20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc47000d150 tx=0x7fc47002c7c0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.028+0000 7fc47e189640 1 -- 172.21.15.79:0/3675775967 shutdown_connections 2024-04-26T19:51:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.028+0000 7fc47e189640 1 --2- 172.21.15.79:0/3675775967 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 0x7fc478071d20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.028+0000 7fc47e189640 1 --2- 172.21.15.79:0/3675775967 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 0x7fc478073bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.028+0000 7fc47e189640 1 -- 172.21.15.79:0/3675775967 >> 172.21.15.79:0/3675775967 conn(0x7fc4780b2540 msgr2=0x7fc4780b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.028+0000 7fc47e189640 1 -- 172.21.15.79:0/3675775967 shutdown_connections 2024-04-26T19:51:30.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.028+0000 7fc47e189640 1 -- 172.21.15.79:0/3675775967 wait complete. 2024-04-26T19:51:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.029+0000 7fc47e189640 1 Processor -- start 2024-04-26T19:51:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.029+0000 7fc47e189640 1 -- start start 2024-04-26T19:51:30.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc47e189640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 0x7fc47814b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc47e189640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 0x7fc4781548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc47e189640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc47815c1e0 0x7fc4781e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc47e189640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc4780cde10 con 0x7fc4780718f0 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc47e189640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc4780cdc90 con 0x7fc47815c1e0 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc47e189640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc4780cdf90 con 0x7fc478070030 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc476ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 0x7fc4781548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc476ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 0x7fc4781548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35004/0 (socket says 172.21.15.79:35004) 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc476ffd640 1 -- 172.21.15.79:0/3901875239 learned_addr learned my addr 172.21.15.79:0/3901875239 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc477fff640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc47815c1e0 0x7fc4781e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc4777fe640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 0x7fc47814b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc476ffd640 1 -- 172.21.15.79:0/3901875239 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 msgr2=0x7fc47814b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc476ffd640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 0x7fc47814b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc476ffd640 1 -- 172.21.15.79:0/3901875239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc47815c1e0 msgr2=0x7fc4781e6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc476ffd640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc47815c1e0 0x7fc4781e6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc476ffd640 1 -- 172.21.15.79:0/3901875239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47815e8f0 con 0x7fc4780718f0 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc476ffd640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 0x7fc4781548f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00c7f0 tx=0x7fc46c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc45ffff640 1 -- 172.21.15.79:0/3901875239 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c015070 con 0x7fc4780718f0 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc45ffff640 1 -- 172.21.15.79:0/3901875239 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46c007a30 con 0x7fc4780718f0 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47815ebd0 con 0x7fc4780718f0 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.030+0000 7fc45ffff640 1 -- 172.21.15.79:0/3901875239 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46c007e00 con 0x7fc4780718f0 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.031+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc478159870 con 0x7fc4780718f0 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.032+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc478159a70 con 0x7fc4780718f0 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.032+0000 7fc45ffff640 1 -- 172.21.15.79:0/3901875239 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc46c018070 con 0x7fc4780718f0 2024-04-26T19:51:30.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.032+0000 7fc45ffff640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc46406c8f0 0x7fc46406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.032+0000 7fc4777fe640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc46406c8f0 0x7fc46406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.032+0000 7fc45ffff640 1 -- 172.21.15.79:0/3901875239 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc46c08be90 con 0x7fc4780718f0 2024-04-26T19:51:30.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.033+0000 7fc4777fe640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc46406c8f0 0x7fc46406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc468006fd0 tx=0x7fc468008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.036+0000 7fc45ffff640 1 -- 172.21.15.79:0/3901875239 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc478159a70 con 0x7fc4780718f0 2024-04-26T19:51:30.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.209+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc4780cda70 con 0x7fc4780718f0 2024-04-26T19:51:30.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.209+0000 7fc45ffff640 1 -- 172.21.15.79:0/3901875239 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fc46c059f50 con 0x7fc4780718f0 2024-04-26T19:51:30.210 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc46406c8f0 msgr2=0x7fc46406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc46406c8f0 0x7fc46406edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc468006fd0 tx=0x7fc468008040 comp rx=0 tx=0).stop 2024-04-26T19:51:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 msgr2=0x7fc4781548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 0x7fc4781548f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc46c00c7f0 tx=0x7fc46c00ccb0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 shutdown_connections 2024-04-26T19:51:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc46406c8f0 0x7fc46406edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc47815c1e0 0x7fc4781e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc4780718f0 0x7fc4781548f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 --2- 172.21.15.79:0/3901875239 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc478070030 0x7fc47814b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 >> 172.21.15.79:0/3901875239 conn(0x7fc4780b2540 msgr2=0x7fc478156350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:30.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 shutdown_connections 2024-04-26T19:51:30.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.211+0000 7fc47e189640 1 -- 172.21.15.79:0/3901875239 wait complete. 2024-04-26T19:51:30.223 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:51:30.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.322+0000 7f31780f9640 1 Processor -- start 2024-04-26T19:51:30.323 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.322+0000 7f31780f9640 1 -- start start 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.323+0000 7f31780f9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f31700b8f60 0x7f317006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.323+0000 7f31780f9640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3170163e10 con 0x7f31700d1830 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.323+0000 7f31780f9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31700d1dd0 con 0x7f3170161ee0 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.323+0000 7f31780f9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f31700d2950 con 0x7f31700b8f60 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.323+0000 7f3175694640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f31700d1830 0x7f3170162ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41494/0 (socket says 172.21.15.79:41494) 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.323+0000 7f3175694640 1 -- 172.21.15.79:0/782787061 learned_addr learned my addr 172.21.15.79:0/782787061 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2085443348 0 0) 0x7f3170163e10 con 0x7f31700d1830 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3175e95640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f31700b8f60 0x7f317006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f315c003520 con 0x7f31700d1830 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2140567980 0 0) 0x7f315c003520 con 0x7f31700d1830 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f31700b8f60 msgr2=0x7f317006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:30.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f31700b8f60 0x7f317006c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 >> v1:172.21.15.83:6789/0 conn(0x7f3170161ee0 legacy=0x7f3170163700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31700d20b0 con 0x7f31700d1830 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f3160001670 con 0x7f31700d1830 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 >> v1:172.21.15.79:6789/0 conn(0x7f31700d1830 legacy=0x7f3170162ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c0039e0 0x7f315c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f315c004800 0x7f315c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f31700b8f60 0x7f315c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f31700d2950 con 0x7f315c004800 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f315c00bb00 con 0x7f315c0039e0 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f315c00bd00 con 0x7f31700b8f60 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3175694640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f315c004800 0x7f315c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3175e95640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c0039e0 0x7f315c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3176696640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f31700b8f60 0x7f315c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3175694640 1 -- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f31700b8f60 msgr2=0x7f315c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3175694640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f31700b8f60 0x7f315c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3175694640 1 -- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c0039e0 msgr2=0x7f315c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3175694640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c0039e0 0x7f315c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3175694640 1 -- 172.21.15.79:0/782787061 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315c003520 con 0x7f315c004800 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.324+0000 7f3175e95640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c0039e0 0x7f315c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.325+0000 7f3175694640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f315c004800 0x7f315c004c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f316000c210 tx=0x7f315c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbba269d85948fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.325+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160007cf0 con 0x7f315c004800 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.325+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3160005ad0 con 0x7f315c004800 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.325+0000 7f3174e93640 1 -- 172.21.15.79:0/782787061 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160007070 con 0x7f315c004800 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.326+0000 7f31780f9640 1 -- 172.21.15.79:0/782787061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f315c004800 msgr2=0x7f315c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.326+0000 7f31780f9640 1 --2- 172.21.15.79:0/782787061 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f315c004800 0x7f315c004c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f316000c210 tx=0x7f315c005830 comp rx=0 tx=0).stop 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.326+0000 7f3176696640 1 -- 172.21.15.79:0/782787061 reap_dead start 2024-04-26T19:51:30.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.326+0000 7f31780f9640 1 -- 172.21.15.79:0/782787061 shutdown_connections 2024-04-26T19:51:30.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.326+0000 7f31780f9640 1 -- 172.21.15.79:0/782787061 >> 172.21.15.79:0/782787061 conn(0x7f3170073b40 msgr2=0x7f3170073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:30.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.326+0000 7f31780f9640 1 -- 172.21.15.79:0/782787061 shutdown_connections 2024-04-26T19:51:30.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.326+0000 7f31780f9640 1 -- 172.21.15.79:0/782787061 wait complete. 2024-04-26T19:51:30.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.327+0000 7f31780f9640 1 Processor -- start 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.327+0000 7f31780f9640 1 -- start start 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31780f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c004800 0x7f31700b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31780f9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f317013e6a0 0x7f3170152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31780f9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3170153070 0x7f3170141010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31780f9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f317006d010 con 0x7f317013e6a0 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31780f9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f31700d2670 con 0x7f315c004800 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31780f9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f317006d190 con 0x7f3170153070 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175694640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f317013e6a0 0x7f3170152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175694640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f317013e6a0 0x7f3170152b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35026/0 (socket says 172.21.15.79:35026) 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175694640 1 -- 172.21.15.79:0/1939985618 learned_addr learned my addr 172.21.15.79:0/1939985618 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175e95640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c004800 0x7f31700b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3176696640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3170153070 0x7f3170141010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175694640 1 -- 172.21.15.79:0/1939985618 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3170153070 msgr2=0x7f3170141010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175694640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3170153070 0x7f3170141010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175694640 1 -- 172.21.15.79:0/1939985618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c004800 msgr2=0x7f31700b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175694640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c004800 0x7f31700b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175694640 1 -- 172.21.15.79:0/1939985618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31700b6e40 con 0x7f317013e6a0 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175e95640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c004800 0x7f31700b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f3175694640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f317013e6a0 0x7f3170152b30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f316000c210 tx=0x7f31700d2280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31667fc640 1 -- 172.21.15.79:0/1939985618 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3160001630 con 0x7f317013e6a0 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31667fc640 1 -- 172.21.15.79:0/1939985618 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3160038040 con 0x7f317013e6a0 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31700b7120 con 0x7f317013e6a0 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31667fc640 1 -- 172.21.15.79:0/1939985618 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31600072b0 con 0x7f317013e6a0 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.328+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31700b76e0 con 0x7f317013e6a0 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.329+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31700506c0 con 0x7f317013e6a0 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.330+0000 7f31667fc640 1 -- 172.21.15.79:0/1939985618 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3160007450 con 0x7f317013e6a0 2024-04-26T19:51:30.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.330+0000 7f31667fc640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f315406c920 0x7f315406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.330+0000 7f31667fc640 1 -- 172.21.15.79:0/1939985618 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f31600ac1a0 con 0x7f317013e6a0 2024-04-26T19:51:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.330+0000 7f3175e95640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f315406c920 0x7f315406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.331+0000 7f3175e95640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f315406c920 0x7f315406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f317006f5d0 tx=0x7f316c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.335+0000 7f31667fc640 1 -- 172.21.15.79:0/1939985618 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f316007a1d0 con 0x7f317013e6a0 2024-04-26T19:51:30.424 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:51:30.426 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:51:30.515 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.515+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f31700b4080 con 0x7f317013e6a0 2024-04-26T19:51:30.516 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.515+0000 7f31667fc640 1 -- 172.21.15.79:0/1939985618 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f3160079ad0 con 0x7f317013e6a0 2024-04-26T19:51:30.516 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:51:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f315406c920 msgr2=0x7f315406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f315406c920 0x7f315406ede0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f317006f5d0 tx=0x7f316c00b040 comp rx=0 tx=0).stop 2024-04-26T19:51:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f317013e6a0 msgr2=0x7f3170152b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f317013e6a0 0x7f3170152b30 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f316000c210 tx=0x7f31700d2280 comp rx=0 tx=0).stop 2024-04-26T19:51:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 shutdown_connections 2024-04-26T19:51:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f315406c920 0x7f315406ede0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.518 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3170153070 0x7f3170141010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f317013e6a0 0x7f3170152b30 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 --2- 172.21.15.79:0/1939985618 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f315c004800 0x7f31700b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 >> 172.21.15.79:0/1939985618 conn(0x7f3170073b40 msgr2=0x7f317006fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 shutdown_connections 2024-04-26T19:51:30.519 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.517+0000 7f31780f9640 1 -- 172.21.15.79:0/1939985618 wait complete. 2024-04-26T19:51:30.529 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-26T19:51:30.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.627+0000 7f36ab083640 1 Processor -- start 2024-04-26T19:51:30.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.628+0000 7f36ab083640 1 -- start start 2024-04-26T19:51:30.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.628+0000 7f36ab083640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f36a415ac60 0x7f36a406fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.628+0000 7f36ab083640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a41610c0 con 0x7f36a4071f60 2024-04-26T19:51:30.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.628+0000 7f36ab083640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36a41613a0 con 0x7f36a40701d0 2024-04-26T19:51:30.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.628+0000 7f36ab083640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f36a4161d40 con 0x7f36a415ac60 2024-04-26T19:51:30.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a9620640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f36a4071f60 0x7f36a415ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41506/0 (socket says 172.21.15.79:41506) 2024-04-26T19:51:30.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a9620640 1 -- 172.21.15.79:0/894041563 learned_addr learned my addr 172.21.15.79:0/894041563 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:30.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a8e1f640 1 --2- 172.21.15.79:0/894041563 >> v2:172.21.15.83:3300/0 conn(0x7f36a415ac60 0x7f36a406fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 600627691 0 0) 0x7f36a41610c0 con 0x7f36a4071f60 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3694003520 con 0x7f36a4071f60 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 949917745 0 0) 0x7f3694003520 con 0x7f36a4071f60 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 >> v2:172.21.15.83:3300/0 conn(0x7f36a415ac60 msgr2=0x7f36a406fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 --2- 172.21.15.79:0/894041563 >> v2:172.21.15.83:3300/0 conn(0x7f36a415ac60 0x7f36a406fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 >> v1:172.21.15.83:6789/0 conn(0x7f36a40701d0 legacy=0x7f36a4071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a4161680 con 0x7f36a4071f60 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f369c0019f0 con 0x7f36a4071f60 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 >> v1:172.21.15.79:6789/0 conn(0x7f36a4071f60 legacy=0x7f36a415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36940039b0 0x7f3694003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3694004800 0x7f3694004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f369400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4161d40 con 0x7f36940051c0 2024-04-26T19:51:30.633 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f369400c060 con 0x7f36940039b0 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f369400c260 con 0x7f3694004800 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a9620640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f369400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a8e1f640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36940039b0 0x7f3694003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a3fff640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3694004800 0x7f3694004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.629+0000 7f36a9620640 1 -- 172.21.15.79:0/894041563 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3694004800 msgr2=0x7f3694004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.630+0000 7f36a9620640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3694004800 0x7f3694004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.630+0000 7f36a9620640 1 -- 172.21.15.79:0/894041563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36940039b0 msgr2=0x7f3694003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.630+0000 7f36a9620640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36940039b0 0x7f3694003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.630+0000 7f36a9620640 1 -- 172.21.15.79:0/894041563 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3694003520 con 0x7f36940051c0 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.630+0000 7f36a9620640 1 -- 172.21.15.79:0/894041563 reap_dead start 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.630+0000 7f36a9620640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f369400b9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f369c00a840 tx=0x7f3694005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4cad0706b4d4d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.630+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c00a460 con 0x7f36940051c0 2024-04-26T19:51:30.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.630+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369c02f070 con 0x7f36940051c0 2024-04-26T19:51:30.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.630+0000 7f36a37fe640 1 -- 172.21.15.79:0/894041563 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c02bcb0 con 0x7f36940051c0 2024-04-26T19:51:30.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.631+0000 7f36ab083640 1 -- 172.21.15.79:0/894041563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 msgr2=0x7f369400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.631+0000 7f36ab083640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f369400b9c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f369c00a840 tx=0x7f3694005830 comp rx=0 tx=0).stop 2024-04-26T19:51:30.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.631+0000 7f36ab083640 1 -- 172.21.15.79:0/894041563 shutdown_connections 2024-04-26T19:51:30.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.631+0000 7f36ab083640 1 --2- 172.21.15.79:0/894041563 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f369400b9c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.631+0000 7f36ab083640 1 -- 172.21.15.79:0/894041563 >> 172.21.15.79:0/894041563 conn(0x7f36a40b3ed0 msgr2=0x7f36a40681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:30.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.631+0000 7f36ab083640 1 -- 172.21.15.79:0/894041563 shutdown_connections 2024-04-26T19:51:30.635 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.631+0000 7f36ab083640 1 -- 172.21.15.79:0/894041563 wait complete. 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.632+0000 7f36ab083640 1 Processor -- start 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.632+0000 7f36ab083640 1 -- start start 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36ab083640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f36a414d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36ab083640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f36a414e210 0x7f36a414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36ab083640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a415fdb0 0x7f36a4160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36ab083640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f36a415b040 con 0x7f36940051c0 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36ab083640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f36a4160d90 con 0x7f36a415fdb0 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36ab083640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f36a415b1c0 con 0x7f36a414e210 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a8e1f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f36a414d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a8e1f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f36a414d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35040/0 (socket says 172.21.15.79:35040) 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a8e1f640 1 -- 172.21.15.79:0/1525385298 learned_addr learned my addr 172.21.15.79:0/1525385298 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:30.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a9620640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a415fdb0 0x7f36a4160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a3fff640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f36a414e210 0x7f36a414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a8e1f640 1 -- 172.21.15.79:0/1525385298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f36a414e210 msgr2=0x7f36a414c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a8e1f640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f36a414e210 0x7f36a414c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a8e1f640 1 -- 172.21.15.79:0/1525385298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a415fdb0 msgr2=0x7f36a4160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a8e1f640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a415fdb0 0x7f36a4160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a8e1f640 1 -- 172.21.15.79:0/1525385298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a41539b0 con 0x7f36940051c0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.633+0000 7f36a8e1f640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f36a414d8a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f36980118e0 tx=0x7f3698011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.634+0000 7f36a17fa640 1 -- 172.21.15.79:0/1525385298 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369800faf0 con 0x7f36940051c0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.634+0000 7f36a17fa640 1 -- 172.21.15.79:0/1525385298 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f369800fc90 con 0x7f36940051c0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.634+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a414e710 con 0x7f36940051c0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.634+0000 7f36a17fa640 1 -- 172.21.15.79:0/1525385298 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698014440 con 0x7f36940051c0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.634+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a4153f50 con 0x7f36940051c0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.635+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a40506c0 con 0x7f36940051c0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.635+0000 7f36a17fa640 1 -- 172.21.15.79:0/1525385298 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3698012040 con 0x7f36940051c0 2024-04-26T19:51:30.640 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.635+0000 7f36a17fa640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f368806c8f0 0x7f368806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.636+0000 7f36a17fa640 1 -- 172.21.15.79:0/1525385298 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f369808c670 con 0x7f36940051c0 2024-04-26T19:51:30.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.636+0000 7f36a3fff640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f368806c8f0 0x7f368806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.636+0000 7f36a3fff640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f368806c8f0 0x7f368806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3690000c00 tx=0x7f3690009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.640+0000 7f36a17fa640 1 -- 172.21.15.79:0/1525385298 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3698019030 con 0x7f36940051c0 2024-04-26T19:51:30.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.815+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f36a40aa4c0 con 0x7f36940051c0 2024-04-26T19:51:30.816 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.815+0000 7f36a17fa640 1 -- 172.21.15.79:0/1525385298 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f369805a6a0 con 0x7f36940051c0 2024-04-26T19:51:30.816 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:51:30.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f368806c8f0 msgr2=0x7f368806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f368806c8f0 0x7f368806edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3690000c00 tx=0x7f3690009040 comp rx=0 tx=0).stop 2024-04-26T19:51:30.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 msgr2=0x7f36a414d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f36a414d8a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f36980118e0 tx=0x7f3698011da0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 shutdown_connections 2024-04-26T19:51:30.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f368806c8f0 0x7f368806edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f36a415fdb0 0x7f36a4160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f36a414e210 0x7f36a414c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 --2- 172.21.15.79:0/1525385298 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f36940051c0 0x7f36a414d8a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 >> 172.21.15.79:0/1525385298 conn(0x7f36a40b3ed0 msgr2=0x7f36a4159590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:30.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 shutdown_connections 2024-04-26T19:51:30.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.817+0000 7f36ab083640 1 -- 172.21.15.79:0/1525385298 wait complete. 2024-04-26T19:51:30.829 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:51:30.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.925+0000 7f7136415640 1 Processor -- start 2024-04-26T19:51:30.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.925+0000 7f7136415640 1 -- start start 2024-04-26T19:51:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.926+0000 7f7136415640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7130071f20 0x7f7130166850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.926+0000 7f7136415640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71300723d0 con 0x7f713015b850 2024-04-26T19:51:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.926+0000 7f7136415640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71300cf310 con 0x7f71300ced80 2024-04-26T19:51:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.926+0000 7f7136415640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f71300cf5f0 con 0x7f7130071f20 2024-04-26T19:51:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.926+0000 7f712ffff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f713015b850 0x7f71300d03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41516/0 (socket says 172.21.15.79:41516) 2024-04-26T19:51:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.926+0000 7f712ffff640 1 -- 172.21.15.79:0/2535398455 learned_addr learned my addr 172.21.15.79:0/2535398455 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.926+0000 7f71349b2640 1 --2- 172.21.15.79:0/2535398455 >> v2:172.21.15.83:3300/0 conn(0x7f7130071f20 0x7f7130166850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.926+0000 7f712effd640 1 -- 172.21.15.79:0/2535398455 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 198508504 0 0) 0x7f7120001110 con 0x7f713015b850 2024-04-26T19:51:30.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.927+0000 7f712effd640 1 -- 172.21.15.79:0/2535398455 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7118003440 con 0x7f713015b850 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.927+0000 7f712effd640 1 -- 172.21.15.79:0/2535398455 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3536719284 0 0) 0x7f71300cf310 con 0x7f71300ced80 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.927+0000 7f712effd640 1 -- 172.21.15.79:0/2535398455 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7120001110 con 0x7f71300ced80 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.927+0000 7f71349b2640 1 -- 172.21.15.79:0/2535398455 >> v1:172.21.15.83:6789/0 conn(0x7f71300ced80 legacy=0x7f7130071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.927+0000 7f712effd640 1 -- 172.21.15.79:0/2535398455 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 308959190 0 0) 0x7f7118003440 con 0x7f713015b850 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.927+0000 7f71349b2640 1 -- 172.21.15.79:0/2535398455 >> v1:172.21.15.79:6789/0 conn(0x7f713015b850 legacy=0x7f71300d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.927+0000 7f71349b2640 1 -- 172.21.15.79:0/2535398455 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71300cf8d0 con 0x7f7130071f20 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.927+0000 7f71349b2640 1 --2- 172.21.15.79:0/2535398455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7130071f20 0x7f7130166850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7128009d90 tx=0x7f712802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf2d2d64e14c9f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f712effd640 1 -- 172.21.15.79:0/2535398455 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120001840 con 0x7f7130071f20 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f712effd640 1 -- 172.21.15.79:0/2535398455 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7128007540 con 0x7f7130071f20 2024-04-26T19:51:30.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f712effd640 1 -- 172.21.15.79:0/2535398455 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71280078f0 con 0x7f7130071f20 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f7136415640 1 -- 172.21.15.79:0/2535398455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7130071f20 msgr2=0x7f7130166850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f7136415640 1 --2- 172.21.15.79:0/2535398455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7130071f20 0x7f7130166850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7128009d90 tx=0x7f712802a5f0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f7136415640 1 -- 172.21.15.79:0/2535398455 shutdown_connections 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f7136415640 1 --2- 172.21.15.79:0/2535398455 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7130071f20 0x7f7130166850 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f7136415640 1 -- 172.21.15.79:0/2535398455 >> 172.21.15.79:0/2535398455 conn(0x7f71300b39d0 msgr2=0x7f713006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f7136415640 1 -- 172.21.15.79:0/2535398455 shutdown_connections 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.928+0000 7f7136415640 1 -- 172.21.15.79:0/2535398455 wait complete. 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.929+0000 7f7136415640 1 Processor -- start 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.929+0000 7f7136415640 1 -- start start 2024-04-26T19:51:30.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f7136415640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7130071f20 0x7f713015ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f7136415640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300ced80 0x7f71301f4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f7136415640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f713015b850 0x7f7130143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f7136415640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f71300cd9d0 con 0x7f7130071f20 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f7136415640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71300cfe90 con 0x7f71300ced80 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f7136415640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f713015bc30 con 0x7f713015b850 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f712ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7130071f20 0x7f713015ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f712ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7130071f20 0x7f713015ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35042/0 (socket says 172.21.15.79:35042) 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f712ffff640 1 -- 172.21.15.79:0/319942040 learned_addr learned my addr 172.21.15.79:0/319942040 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f712f7fe640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300ced80 0x7f71301f4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.930+0000 7f71349b2640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f713015b850 0x7f7130143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f712ffff640 1 -- 172.21.15.79:0/319942040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f713015b850 msgr2=0x7f7130143ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f712ffff640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f713015b850 0x7f7130143ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f712ffff640 1 -- 172.21.15.79:0/319942040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300ced80 msgr2=0x7f71301f4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f712ffff640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300ced80 0x7f71301f4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:30.933 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f712ffff640 1 -- 172.21.15.79:0/319942040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713015c260 con 0x7f7130071f20 2024-04-26T19:51:30.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f712ffff640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7130071f20 0x7f713015ac60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f71200019b0 tx=0x7f712000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f712cff9640 1 -- 172.21.15.79:0/319942040 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120017070 con 0x7f7130071f20 2024-04-26T19:51:30.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f712cff9640 1 -- 172.21.15.79:0/319942040 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7120005b10 con 0x7f7130071f20 2024-04-26T19:51:30.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713015a470 con 0x7f7130071f20 2024-04-26T19:51:30.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f712cff9640 1 -- 172.21.15.79:0/319942040 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120007070 con 0x7f7130071f20 2024-04-26T19:51:30.934 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.931+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71301560b0 con 0x7f7130071f20 2024-04-26T19:51:30.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.932+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71300506c0 con 0x7f7130071f20 2024-04-26T19:51:30.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.932+0000 7f712cff9640 1 -- 172.21.15.79:0/319942040 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f712001a070 con 0x7f7130071f20 2024-04-26T19:51:30.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.933+0000 7f712cff9640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f711c06c8f0 0x7f711c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:30.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.933+0000 7f712f7fe640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f711c06c8f0 0x7f711c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:30.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.933+0000 7f712cff9640 1 -- 172.21.15.79:0/319942040 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f712008b430 con 0x7f7130071f20 2024-04-26T19:51:30.935 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.933+0000 7f712f7fe640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f711c06c8f0 0x7f711c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f71300afb00 tx=0x7f7124009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:30.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:30.937+0000 7f712cff9640 1 -- 172.21.15.79:0/319942040 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71300723d0 con 0x7f7130071f20 2024-04-26T19:51:31.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.149+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7130073c80 con 0x7f7130071f20 2024-04-26T19:51:31.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.149+0000 7f712cff9640 1 -- 172.21.15.79:0/319942040 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f7120058ea0 con 0x7f7130071f20 2024-04-26T19:51:31.150 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:31.151 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:52.517190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:51:31.151 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2024-04-26T19:51:31.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.151+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f711c06c8f0 msgr2=0x7f711c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.151+0000 7f7136415640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f711c06c8f0 0x7f711c06edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f71300afb00 tx=0x7f7124009040 comp rx=0 tx=0).stop 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.151+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7130071f20 msgr2=0x7f713015ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.151+0000 7f7136415640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7130071f20 0x7f713015ac60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f71200019b0 tx=0x7f712000cb70 comp rx=0 tx=0).stop 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.152+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 shutdown_connections 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.152+0000 7f7136415640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f711c06c8f0 0x7f711c06edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.152+0000 7f7136415640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f713015b850 0x7f7130143ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.152+0000 7f7136415640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71300ced80 0x7f71301f4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.152+0000 7f7136415640 1 --2- 172.21.15.79:0/319942040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7130071f20 0x7f713015ac60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.152+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 >> 172.21.15.79:0/319942040 conn(0x7f71300b39d0 msgr2=0x7f71300afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.152+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 shutdown_connections 2024-04-26T19:51:31.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.152+0000 7f7136415640 1 -- 172.21.15.79:0/319942040 wait complete. 2024-04-26T19:51:31.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:31.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.260+0000 7fa21fb9d640 1 Processor -- start 2024-04-26T19:51:31.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.260+0000 7fa21fb9d640 1 -- start start 2024-04-26T19:51:31.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21fb9d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 0x7fa218163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21fb9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa218159c70 0x7fa21815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21fb9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa2180b3b40 0x7fa2180b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21fb9d640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fa2180b44f0 con 0x7fa218159c70 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21fb9d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa218160280 con 0x7fa2180b3b40 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21fb9d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa218160560 con 0x7fa218159890 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21d138640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa218159c70 0x7fa21815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21d138640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa218159c70 0x7fa21815fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35058/0 (socket says 172.21.15.79:35058) 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21d138640 1 -- 172.21.15.79:0/942320874 learned_addr learned my addr 172.21.15.79:0/942320874 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21e13a640 1 --2- 172.21.15.79:0/942320874 >> v2:172.21.15.83:3300/0 conn(0x7fa2180b3b40 0x7fa2180b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21d939640 1 --2- 172.21.15.79:0/942320874 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 0x7fa218163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21d138640 1 -- 172.21.15.79:0/942320874 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 msgr2=0x7fa218163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21d138640 1 --2- 172.21.15.79:0/942320874 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 0x7fa218163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21d138640 1 -- 172.21.15.79:0/942320874 >> v2:172.21.15.83:3300/0 conn(0x7fa2180b3b40 msgr2=0x7fa2180b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21d138640 1 --2- 172.21.15.79:0/942320874 >> v2:172.21.15.83:3300/0 conn(0x7fa2180b3b40 0x7fa2180b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.261+0000 7fa21d138640 1 -- 172.21.15.79:0/942320874 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa218160840 con 0x7fa218159c70 2024-04-26T19:51:31.264 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21d138640 1 --2- 172.21.15.79:0/942320874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 0x7fa21815fa20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa2180b1d30 tx=0x7fa20802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3539f9f08da90aea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21c937640 1 -- 172.21.15.79:0/942320874 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208037070 con 0x7fa218159c70 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21c937640 1 -- 172.21.15.79:0/942320874 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa208032440 con 0x7fa218159c70 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21c937640 1 -- 172.21.15.79:0/942320874 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208032810 con 0x7fa218159c70 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21fb9d640 1 -- 172.21.15.79:0/942320874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 msgr2=0x7fa21815fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/942320874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 0x7fa21815fa20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa2180b1d30 tx=0x7fa20802a1e0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21fb9d640 1 -- 172.21.15.79:0/942320874 shutdown_connections 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/942320874 >> v2:172.21.15.83:3300/0 conn(0x7fa2180b3b40 0x7fa2180b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/942320874 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 0x7fa21815fa20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.262+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/942320874 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 0x7fa218163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.263+0000 7fa21fb9d640 1 -- 172.21.15.79:0/942320874 >> 172.21.15.79:0/942320874 conn(0x7fa2180b2720 msgr2=0x7fa218070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.263+0000 7fa21fb9d640 1 -- 172.21.15.79:0/942320874 shutdown_connections 2024-04-26T19:51:31.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.263+0000 7fa21fb9d640 1 -- 172.21.15.79:0/942320874 wait complete. 2024-04-26T19:51:31.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.264+0000 7fa21fb9d640 1 Processor -- start 2024-04-26T19:51:31.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.264+0000 7fa21fb9d640 1 -- start start 2024-04-26T19:51:31.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.264+0000 7fa21fb9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2180b3b40 0x7fa21815a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.264+0000 7fa21fb9d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 0x7fa218141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.264+0000 7fa21fb9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 0x7fa2181e8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.264+0000 7fa21fb9d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa21815ff60 con 0x7fa218159c70 2024-04-26T19:51:31.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.264+0000 7fa21fb9d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa218160b20 con 0x7fa2180b3b40 2024-04-26T19:51:31.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.264+0000 7fa21fb9d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa21806c350 con 0x7fa218159890 2024-04-26T19:51:31.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21e13a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 0x7fa2181e8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21e13a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 0x7fa2181e8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35070/0 (socket says 172.21.15.79:35070) 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21e13a640 1 -- 172.21.15.79:0/2884197990 learned_addr learned my addr 172.21.15.79:0/2884197990 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21d939640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2180b3b40 0x7fa21815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21d138640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 0x7fa218141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21e13a640 1 -- 172.21.15.79:0/2884197990 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 msgr2=0x7fa218141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21e13a640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 0x7fa218141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21e13a640 1 -- 172.21.15.79:0/2884197990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2180b3b40 msgr2=0x7fa21815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21e13a640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2180b3b40 0x7fa21815a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21e13a640 1 -- 172.21.15.79:0/2884197990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa218141c50 con 0x7fa218159c70 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21e13a640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 0x7fa2181e8530 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa21000ef00 tx=0x7fa21000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa20e7fc640 1 -- 172.21.15.79:0/2884197990 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210018020 con 0x7fa218159c70 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa20e7fc640 1 -- 172.21.15.79:0/2884197990 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa210014400 con 0x7fa218159c70 2024-04-26T19:51:31.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa218142020 con 0x7fa218159c70 2024-04-26T19:51:31.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa20e7fc640 1 -- 172.21.15.79:0/2884197990 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2100147f0 con 0x7fa218159c70 2024-04-26T19:51:31.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.265+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2181425e0 con 0x7fa218159c70 2024-04-26T19:51:31.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.266+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2180506c0 con 0x7fa218159c70 2024-04-26T19:51:31.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.267+0000 7fa20e7fc640 1 -- 172.21.15.79:0/2884197990 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa210005d10 con 0x7fa218159c70 2024-04-26T19:51:31.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.267+0000 7fa20e7fc640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1fc06c8f0 0x7fa1fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.267+0000 7fa21d939640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1fc06c8f0 0x7fa1fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.267+0000 7fa20e7fc640 1 -- 172.21.15.79:0/2884197990 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa21008c490 con 0x7fa218159c70 2024-04-26T19:51:31.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.268+0000 7fa21d939640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1fc06c8f0 0x7fa1fc06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa214009b40 tx=0x7fa214009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:31.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.272+0000 7fa20e7fc640 1 -- 172.21.15.79:0/2884197990 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa210022070 con 0x7fa218159c70 2024-04-26T19:51:31.450 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.449+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa2180af3b0 con 0x7fa218159c70 2024-04-26T19:51:31.451 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.451+0000 7fa20e7fc640 1 -- 172.21.15.79:0/2884197990 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7fa210059e70 con 0x7fa218159c70 2024-04-26T19:51:31.451 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:31.452 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:31.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1fc06c8f0 msgr2=0x7fa1fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1fc06c8f0 0x7fa1fc06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa214009b40 tx=0x7fa214009190 comp rx=0 tx=0).stop 2024-04-26T19:51:31.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 msgr2=0x7fa2181e8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 0x7fa2181e8530 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa21000ef00 tx=0x7fa21000b480 comp rx=0 tx=0).stop 2024-04-26T19:51:31.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 shutdown_connections 2024-04-26T19:51:31.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1fc06c8f0 0x7fa1fc06edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa218159c70 0x7fa2181e8530 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa218159890 0x7fa218141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.454 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 --2- 172.21.15.79:0/2884197990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa2180b3b40 0x7fa21815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 >> 172.21.15.79:0/2884197990 conn(0x7fa2180b2720 msgr2=0x7fa21815e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:31.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 shutdown_connections 2024-04-26T19:51:31.455 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.453+0000 7fa21fb9d640 1 -- 172.21.15.79:0/2884197990 wait complete. 2024-04-26T19:51:31.464 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T19:51:31.465 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:51:31.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.560+0000 7f62c07f9640 1 Processor -- start 2024-04-26T19:51:31.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.560+0000 7f62c07f9640 1 -- start start 2024-04-26T19:51:31.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62c07f9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f62b8160770 0x7f62b81637e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62c07f9640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b8144ca0 con 0x7f62b8160390 2024-04-26T19:51:31.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62c07f9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b8163e00 con 0x7f62b80b4220 2024-04-26T19:51:31.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62c07f9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f62b8144f30 con 0x7f62b8160770 2024-04-26T19:51:31.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62be595640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f62b8160390 0x7f62b80b3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41532/0 (socket says 172.21.15.79:41532) 2024-04-26T19:51:31.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62be595640 1 -- 172.21.15.79:0/2777108069 learned_addr learned my addr 172.21.15.79:0/2777108069 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:31.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62bed96640 1 --2- 172.21.15.79:0/2777108069 >> v2:172.21.15.83:3300/0 conn(0x7f62b8160770 0x7f62b81637e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62bd593640 1 -- 172.21.15.79:0/2777108069 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2196590388 0 0) 0x7f62b8144ca0 con 0x7f62b8160390 2024-04-26T19:51:31.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62bd593640 1 -- 172.21.15.79:0/2777108069 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62a4003520 con 0x7f62b8160390 2024-04-26T19:51:31.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62bed96640 1 -- 172.21.15.79:0/2777108069 >> v1:172.21.15.83:6789/0 conn(0x7f62b80b4220 legacy=0x7f62b81630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62bd593640 1 -- 172.21.15.79:0/2777108069 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1275666515 0 0) 0x7f62a4003520 con 0x7f62b8160390 2024-04-26T19:51:31.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62bed96640 1 -- 172.21.15.79:0/2777108069 >> v1:172.21.15.79:6789/0 conn(0x7f62b8160390 legacy=0x7f62b80b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.561+0000 7f62bed96640 1 -- 172.21.15.79:0/2777108069 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b8145210 con 0x7f62b8160770 2024-04-26T19:51:31.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.562+0000 7f62bed96640 1 --2- 172.21.15.79:0/2777108069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8160770 0x7f62b81637e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009d90 tx=0x7f62b002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5a3982418271d80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:31.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.562+0000 7f62bd593640 1 -- 172.21.15.79:0/2777108069 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b4001670 con 0x7f62b8160770 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.562+0000 7f62bd593640 1 -- 172.21.15.79:0/2777108069 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0007540 con 0x7f62b8160770 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.562+0000 7f62bd593640 1 -- 172.21.15.79:0/2777108069 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00078f0 con 0x7f62b8160770 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.563+0000 7f62c07f9640 1 -- 172.21.15.79:0/2777108069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8160770 msgr2=0x7f62b81637e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.563+0000 7f62c07f9640 1 --2- 172.21.15.79:0/2777108069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8160770 0x7f62b81637e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009d90 tx=0x7f62b002a5f0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.563+0000 7f62c07f9640 1 -- 172.21.15.79:0/2777108069 shutdown_connections 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.563+0000 7f62c07f9640 1 --2- 172.21.15.79:0/2777108069 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b8160770 0x7f62b81637e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.563+0000 7f62c07f9640 1 -- 172.21.15.79:0/2777108069 >> 172.21.15.79:0/2777108069 conn(0x7f62b80b2720 msgr2=0x7f62b8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.563+0000 7f62c07f9640 1 -- 172.21.15.79:0/2777108069 shutdown_connections 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.563+0000 7f62c07f9640 1 -- 172.21.15.79:0/2777108069 wait complete. 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.564+0000 7f62c07f9640 1 Processor -- start 2024-04-26T19:51:31.565 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.564+0000 7f62c07f9640 1 -- start start 2024-04-26T19:51:31.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62c07f9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b80b4220 0x7f62b815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62c07f9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f62b8160390 0x7f62b81580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62c07f9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b8160770 0x7f62b80af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62c07f9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8145b90 con 0x7f62b8160770 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62c07f9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f62b8145a10 con 0x7f62b80b4220 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62c07f9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f62b816a470 con 0x7f62b8160390 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bed96640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b8160770 0x7f62b80af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bed96640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b8160770 0x7f62b80af3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35072/0 (socket says 172.21.15.79:35072) 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bed96640 1 -- 172.21.15.79:0/4204631931 learned_addr learned my addr 172.21.15.79:0/4204631931 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62be595640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b80b4220 0x7f62b815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bdd94640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f62b8160390 0x7f62b81580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bed96640 1 -- 172.21.15.79:0/4204631931 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f62b8160390 msgr2=0x7f62b81580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bed96640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f62b8160390 0x7f62b81580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bed96640 1 -- 172.21.15.79:0/4204631931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b80b4220 msgr2=0x7f62b815e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bed96640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b80b4220 0x7f62b815e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bed96640 1 -- 172.21.15.79:0/4204631931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b80afea0 con 0x7f62b8160770 2024-04-26T19:51:31.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62be595640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b80b4220 0x7f62b815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:31.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62bed96640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b8160770 0x7f62b80af3e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f62b002abe0 tx=0x7f62b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:31.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62aeffd640 1 -- 172.21.15.79:0/4204631931 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0040070 con 0x7f62b8160770 2024-04-26T19:51:31.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.565+0000 7f62aeffd640 1 -- 172.21.15.79:0/4204631931 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b0031c40 con 0x7f62b8160770 2024-04-26T19:51:31.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.566+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b8157600 con 0x7f62b8160770 2024-04-26T19:51:31.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.566+0000 7f62aeffd640 1 -- 172.21.15.79:0/4204631931 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0009450 con 0x7f62b8160770 2024-04-26T19:51:31.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.566+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b8157bc0 con 0x7f62b8160770 2024-04-26T19:51:31.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.567+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b8157dc0 con 0x7f62b8160770 2024-04-26T19:51:31.569 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.567+0000 7f62aeffd640 1 -- 172.21.15.79:0/4204631931 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62b0039040 con 0x7f62b8160770 2024-04-26T19:51:31.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.567+0000 7f62aeffd640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f629c06c8f0 0x7f629c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.568+0000 7f62aeffd640 1 -- 172.21.15.79:0/4204631931 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f62b00ad3d0 con 0x7f62b8160770 2024-04-26T19:51:31.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.568+0000 7f62be595640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f629c06c8f0 0x7f629c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.568+0000 7f62be595640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f629c06c8f0 0x7f629c06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005880 tx=0x7f62b4000bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:31.572 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.572+0000 7f62aeffd640 1 -- 172.21.15.79:0/4204631931 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f62b8157dc0 con 0x7f62b8160770 2024-04-26T19:51:31.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.788+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62b80b01e0 con 0x7f62b8160770 2024-04-26T19:51:31.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.788+0000 7f62aeffd640 1 -- 172.21.15.79:0/4204631931 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f62b007ad30 con 0x7f62b8160770 2024-04-26T19:51:31.789 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:31.789 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:52.517190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:51:31.790 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2024-04-26T19:51:31.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f629c06c8f0 msgr2=0x7f629c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f629c06c8f0 0x7f629c06edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f62b4005880 tx=0x7f62b4000bc0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b8160770 msgr2=0x7f62b80af3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b8160770 0x7f62b80af3e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f62b002abe0 tx=0x7f62b0004a40 comp rx=0 tx=0).stop 2024-04-26T19:51:31.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 shutdown_connections 2024-04-26T19:51:31.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f629c06c8f0 0x7f629c06edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f62b8160770 0x7f62b80af3e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f62b8160390 0x7f62b81580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 --2- 172.21.15.79:0/4204631931 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f62b80b4220 0x7f62b815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.790+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 >> 172.21.15.79:0/4204631931 conn(0x7f62b80b2720 msgr2=0x7f62b8160bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:31.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.791+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 shutdown_connections 2024-04-26T19:51:31.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.791+0000 7f62c07f9640 1 -- 172.21.15.79:0/4204631931 wait complete. 2024-04-26T19:51:31.802 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:51:31.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.897+0000 7f2f14f22640 1 Processor -- start 2024-04-26T19:51:31.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.897+0000 7f2f14f22640 1 -- start start 2024-04-26T19:51:31.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.898+0000 7f2f14f22640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2f1006f800 0x7f2f1006fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.898+0000 7f2f14f22640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f100b3e10 con 0x7f2f1008d0f0 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.898+0000 7f2f14f22640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f100d22d0 con 0x7f2f101612d0 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.898+0000 7f2f14f22640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2f100d25b0 con 0x7f2f1006f800 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.898+0000 7f2f0ed9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f2f1008d0f0 0x7f2f1008d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41534/0 (socket says 172.21.15.79:41534) 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.898+0000 7f2f0ed9d640 1 -- 172.21.15.79:0/3067576662 learned_addr learned my addr 172.21.15.79:0/3067576662 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.899+0000 7f2f0dd9b640 1 --2- 172.21.15.79:0/3067576662 >> v2:172.21.15.83:3300/0 conn(0x7f2f1006f800 0x7f2f1006fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.899+0000 7f2f0d59a640 1 -- 172.21.15.79:0/3067576662 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2493581729 0 0) 0x7f2f100b3e10 con 0x7f2f1008d0f0 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.899+0000 7f2f0d59a640 1 -- 172.21.15.79:0/3067576662 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2efc003520 con 0x7f2f1008d0f0 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.899+0000 7f2f0dd9b640 1 -- 172.21.15.79:0/3067576662 >> v1:172.21.15.83:6789/0 conn(0x7f2f101612d0 legacy=0x7f2f101648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.899+0000 7f2f0dd9b640 1 -- 172.21.15.79:0/3067576662 >> v1:172.21.15.79:6789/0 conn(0x7f2f1008d0f0 legacy=0x7f2f1008d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.899+0000 7f2f0dd9b640 1 -- 172.21.15.79:0/3067576662 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f100d2890 con 0x7f2f1006f800 2024-04-26T19:51:31.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f0dd9b640 1 --2- 172.21.15.79:0/3067576662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1006f800 0x7f2f1006fc90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80097a0 tx=0x7f2ef802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f864d101e41c4181 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:31.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f0d59a640 1 -- 172.21.15.79:0/3067576662 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8037070 con 0x7f2f1006f800 2024-04-26T19:51:31.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f0d59a640 1 -- 172.21.15.79:0/3067576662 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef8032440 con 0x7f2f1006f800 2024-04-26T19:51:31.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f0d59a640 1 -- 172.21.15.79:0/3067576662 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80327f0 con 0x7f2f1006f800 2024-04-26T19:51:31.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f14f22640 1 -- 172.21.15.79:0/3067576662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1006f800 msgr2=0x7f2f1006fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f14f22640 1 --2- 172.21.15.79:0/3067576662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1006f800 0x7f2f1006fc90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80097a0 tx=0x7f2ef802a230 comp rx=0 tx=0).stop 2024-04-26T19:51:31.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f14f22640 1 -- 172.21.15.79:0/3067576662 shutdown_connections 2024-04-26T19:51:31.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f14f22640 1 --2- 172.21.15.79:0/3067576662 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1006f800 0x7f2f1006fc90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f14f22640 1 -- 172.21.15.79:0/3067576662 >> 172.21.15.79:0/3067576662 conn(0x7f2f10071c30 msgr2=0x7f2f1006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:31.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f14f22640 1 -- 172.21.15.79:0/3067576662 shutdown_connections 2024-04-26T19:51:31.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.900+0000 7f2f14f22640 1 -- 172.21.15.79:0/3067576662 wait complete. 2024-04-26T19:51:31.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.902+0000 7f2f14f22640 1 Processor -- start 2024-04-26T19:51:31.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.902+0000 7f2f14f22640 1 -- start start 2024-04-26T19:51:31.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.902+0000 7f2f14f22640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f1006f800 0x7f2f101fdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.902+0000 7f2f14f22640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1008d0f0 0x7f2f101ed210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.902+0000 7f2f14f22640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2f101612d0 0x7f2f101f92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.902+0000 7f2f14f22640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2f100d1e80 con 0x7f2f1006f800 2024-04-26T19:51:31.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.902+0000 7f2f14f22640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2f100d2e50 con 0x7f2f1008d0f0 2024-04-26T19:51:31.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.902+0000 7f2f14f22640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f2f100d2000 con 0x7f2f101612d0 2024-04-26T19:51:31.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0e59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f1006f800 0x7f2f101fdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.907 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0e59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f1006f800 0x7f2f101fdab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35076/0 (socket says 172.21.15.79:35076) 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0e59c640 1 -- 172.21.15.79:0/2502853246 learned_addr learned my addr 172.21.15.79:0/2502853246 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0ed9d640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2f101612d0 0x7f2f101f92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0dd9b640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1008d0f0 0x7f2f101ed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0e59c640 1 -- 172.21.15.79:0/2502853246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2f101612d0 msgr2=0x7f2f101f92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0e59c640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2f101612d0 0x7f2f101f92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0e59c640 1 -- 172.21.15.79:0/2502853246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1008d0f0 msgr2=0x7f2f101ed210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0e59c640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1008d0f0 0x7f2f101ed210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0e59c640 1 -- 172.21.15.79:0/2502853246 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f101ed750 con 0x7f2f1006f800 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f0e59c640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f1006f800 0x7f2f101fdab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000c400 tx=0x7f2f0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f06ffd640 1 -- 172.21.15.79:0/2502853246 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f000052d0 con 0x7f2f1006f800 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f06ffd640 1 -- 172.21.15.79:0/2502853246 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f00005490 con 0x7f2f1006f800 2024-04-26T19:51:31.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f101eda00 con 0x7f2f1006f800 2024-04-26T19:51:31.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f06ffd640 1 -- 172.21.15.79:0/2502853246 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00007260 con 0x7f2f1006f800 2024-04-26T19:51:31.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.903+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f101edfc0 con 0x7f2f1006f800 2024-04-26T19:51:31.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.904+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1006f3a0 con 0x7f2f1006f800 2024-04-26T19:51:31.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.905+0000 7f2f06ffd640 1 -- 172.21.15.79:0/2502853246 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2f00007400 con 0x7f2f1006f800 2024-04-26T19:51:31.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.905+0000 7f2f06ffd640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2ef006c8f0 0x7f2ef006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:31.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.905+0000 7f2f06ffd640 1 -- 172.21.15.79:0/2502853246 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2f0008b430 con 0x7f2f1006f800 2024-04-26T19:51:31.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.905+0000 7f2f0dd9b640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2ef006c8f0 0x7f2ef006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:31.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.905+0000 7f2f0dd9b640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2ef006c8f0 0x7f2ef006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8034000 tx=0x7f2ef802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:31.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:31.909+0000 7f2f06ffd640 1 -- 172.21.15.79:0/2502853246 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2f1006f3a0 con 0x7f2f1006f800 2024-04-26T19:51:32.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.120+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f100d3700 con 0x7f2f1006f800 2024-04-26T19:51:32.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.120+0000 7f2f06ffd640 1 -- 172.21.15.79:0/2502853246 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5174 (secure 0 0 0) 0x7f2f00058e10 con 0x7f2f1006f800 2024-04-26T19:51:32.121 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:32.121 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/2992369923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":2992369923},{"type":"v1","addr":"172.21.15.79:6836","nonce":2992369923}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/496421908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":496421908},{"type":"v1","addr":"172.21.15.79:6837","nonce":496421908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4299,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/2524995674","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":2524995674},{"type":"v1","addr":"172.21.15.83:6834","nonce":2524995674}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:50:30.451088+0000","modified":"2024-04-26T19:50:52.517190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4302},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4302":{"gid":4302,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.83:6835/3159271692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":3159271692},{"type":"v1","addr":"172.21.15.83:6835","nonce":3159271692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4302,"qdb_cluster":[4302]},"id":1}]} 2024-04-26T19:51:32.122 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 9 2024-04-26T19:51:32.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.122+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2ef006c8f0 msgr2=0x7f2ef006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.122+0000 7f2f14f22640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2ef006c8f0 0x7f2ef006edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8034000 tx=0x7f2ef802e000 comp rx=0 tx=0).stop 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.122+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f1006f800 msgr2=0x7f2f101fdab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.122+0000 7f2f14f22640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f1006f800 0x7f2f101fdab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000c400 tx=0x7f2f0000c8c0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.122+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 shutdown_connections 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.123+0000 7f2f14f22640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2ef006c8f0 0x7f2ef006edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.123+0000 7f2f14f22640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2f101612d0 0x7f2f101f92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.123+0000 7f2f14f22640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2f1008d0f0 0x7f2f101ed210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.123+0000 7f2f14f22640 1 --2- 172.21.15.79:0/2502853246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2f1006f800 0x7f2f101fdab0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.123+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 >> 172.21.15.79:0/2502853246 conn(0x7f2f10071c30 msgr2=0x7f2f10164400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.123+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 shutdown_connections 2024-04-26T19:51:32.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.123+0000 7f2f14f22640 1 -- 172.21.15.79:0/2502853246 wait complete. 2024-04-26T19:51:32.134 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:32.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.230+0000 7feca8bf0640 1 Processor -- start 2024-04-26T19:51:32.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.230+0000 7feca8bf0640 1 -- start start 2024-04-26T19:51:32.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.231+0000 7feca8bf0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feca4154220 0x7feca40c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.231+0000 7feca8bf0640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca40d0d90 con 0x7feca40c9190 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.231+0000 7feca8bf0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feca40cfcf0 con 0x7feca40b83f0 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.231+0000 7feca8bf0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7feca40cffd0 con 0x7feca4154220 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.231+0000 7feca1d9b640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7feca40c9190 0x7feca40b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41548/0 (socket says 172.21.15.79:41548) 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.231+0000 7feca1d9b640 1 -- 172.21.15.79:0/468408552 learned_addr learned my addr 172.21.15.79:0/468408552 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca259c640 1 --2- 172.21.15.79:0/468408552 >> v2:172.21.15.83:3300/0 conn(0x7feca4154220 0x7feca40c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667864072 0 0) 0x7feca40d0d90 con 0x7feca40c9190 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fec8c003520 con 0x7feca40c9190 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4252396581 0 0) 0x7fec8c003520 con 0x7feca40c9190 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 >> v2:172.21.15.83:3300/0 conn(0x7feca4154220 msgr2=0x7feca40c8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 --2- 172.21.15.79:0/468408552 >> v2:172.21.15.83:3300/0 conn(0x7feca4154220 0x7feca40c8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 >> v1:172.21.15.83:6789/0 conn(0x7feca40b83f0 legacy=0x7feca40d1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40d02b0 con 0x7feca40c9190 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fec98001670 con 0x7feca40c9190 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 >> v1:172.21.15.79:6789/0 conn(0x7feca40c9190 legacy=0x7feca40b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fec8c0039b0 0x7fec8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec8c004800 0x7fec8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec8c0051c0 0x7fec8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feca40cffd0 con 0x7fec8c0051c0 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c00c160 con 0x7fec8c004800 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fec8c00c360 con 0x7fec8c0039b0 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca259c640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fec8c0039b0 0x7fec8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca2d9d640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec8c0051c0 0x7fec8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca1d9b640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec8c004800 0x7fec8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca2d9d640 1 -- 172.21.15.79:0/468408552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fec8c0039b0 msgr2=0x7fec8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca2d9d640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fec8c0039b0 0x7fec8c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca2d9d640 1 -- 172.21.15.79:0/468408552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec8c004800 msgr2=0x7fec8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca2d9d640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fec8c004800 0x7fec8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.232+0000 7feca2d9d640 1 -- 172.21.15.79:0/468408552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c003520 con 0x7fec8c0051c0 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.233+0000 7feca2d9d640 1 -- 172.21.15.79:0/468408552 reap_dead start 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.233+0000 7feca2d9d640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec8c0051c0 0x7fec8c00b9c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fec94001e20 tx=0x7fec8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427622ae29e65b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.233+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94001a70 con 0x7fec8c0051c0 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.233+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec9402ae40 con 0x7fec8c0051c0 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.233+0000 7feca159a640 1 -- 172.21.15.79:0/468408552 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94003bb0 con 0x7fec8c0051c0 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.234+0000 7feca8bf0640 1 -- 172.21.15.79:0/468408552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec8c0051c0 msgr2=0x7fec8c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.234+0000 7feca8bf0640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec8c0051c0 0x7fec8c00b9c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fec94001e20 tx=0x7fec8c005830 comp rx=0 tx=0).stop 2024-04-26T19:51:32.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.234+0000 7feca8bf0640 1 -- 172.21.15.79:0/468408552 shutdown_connections 2024-04-26T19:51:32.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.234+0000 7feca8bf0640 1 --2- 172.21.15.79:0/468408552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fec8c0051c0 0x7fec8c00b9c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.234+0000 7feca8bf0640 1 -- 172.21.15.79:0/468408552 >> 172.21.15.79:0/468408552 conn(0x7feca40741a0 msgr2=0x7feca40b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:32.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.234+0000 7feca8bf0640 1 -- 172.21.15.79:0/468408552 shutdown_connections 2024-04-26T19:51:32.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.234+0000 7feca8bf0640 1 -- 172.21.15.79:0/468408552 wait complete. 2024-04-26T19:51:32.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.235+0000 7feca8bf0640 1 Processor -- start 2024-04-26T19:51:32.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.235+0000 7feca8bf0640 1 -- start start 2024-04-26T19:51:32.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca8bf0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fec8c0051c0 0x7feca415c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca8bf0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca4157680 0x7feca415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca8bf0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feca40b4c50 0x7feca40d2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca8bf0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feca4154600 con 0x7feca40b4c50 2024-04-26T19:51:32.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca8bf0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feca40d0870 con 0x7feca4157680 2024-04-26T19:51:32.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca8bf0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7feca40c8870 con 0x7fec8c0051c0 2024-04-26T19:51:32.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca2d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feca40b4c50 0x7feca40d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca2d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feca40b4c50 0x7feca40d2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35088/0 (socket says 172.21.15.79:35088) 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca2d9d640 1 -- 172.21.15.79:0/2736741283 learned_addr learned my addr 172.21.15.79:0/2736741283 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca1d9b640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca4157680 0x7feca415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca259c640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fec8c0051c0 0x7feca415c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca2d9d640 1 -- 172.21.15.79:0/2736741283 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fec8c0051c0 msgr2=0x7feca415c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca2d9d640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fec8c0051c0 0x7feca415c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca2d9d640 1 -- 172.21.15.79:0/2736741283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca4157680 msgr2=0x7feca415ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca2d9d640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca4157680 0x7feca415ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca2d9d640 1 -- 172.21.15.79:0/2736741283 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca40b54d0 con 0x7feca40b4c50 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca2d9d640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feca40b4c50 0x7feca40d2260 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fec94001f40 tx=0x7fec940014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7fec92ffd640 1 -- 172.21.15.79:0/2736741283 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec94007db0 con 0x7feca40b4c50 2024-04-26T19:51:32.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7fec92ffd640 1 -- 172.21.15.79:0/2736741283 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec94006d60 con 0x7feca40b4c50 2024-04-26T19:51:32.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca41565e0 con 0x7feca40b4c50 2024-04-26T19:51:32.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7fec92ffd640 1 -- 172.21.15.79:0/2736741283 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec940056c0 con 0x7feca40b4c50 2024-04-26T19:51:32.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.236+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca4156ba0 con 0x7feca40b4c50 2024-04-26T19:51:32.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.237+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca4156370 con 0x7feca40b4c50 2024-04-26T19:51:32.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.238+0000 7fec92ffd640 1 -- 172.21.15.79:0/2736741283 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fec94007030 con 0x7feca40b4c50 2024-04-26T19:51:32.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.238+0000 7fec92ffd640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fec8406c920 0x7fec8406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.238+0000 7fec92ffd640 1 -- 172.21.15.79:0/2736741283 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fec940ab950 con 0x7feca40b4c50 2024-04-26T19:51:32.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.238+0000 7feca259c640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fec8406c920 0x7fec8406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.239+0000 7feca259c640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fec8406c920 0x7fec8406ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009a80 tx=0x7fec9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:32.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.242+0000 7fec92ffd640 1 -- 172.21.15.79:0/2736741283 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fec94043060 con 0x7feca40b4c50 2024-04-26T19:51:32.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.421+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feca406d7a0 con 0x7feca40b4c50 2024-04-26T19:51:32.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.422+0000 7fec92ffd640 1 -- 172.21.15.79:0/2736741283 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18852 (secure 0 0 0) 0x7feca40b54d0 con 0x7feca40b4c50 2024-04-26T19:51:32.423 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:32.423 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":27,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:12.230456+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:50:28.469735+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T19:50:29.397999+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-26T19:50:33.118806+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.f","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:32.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fec8406c920 msgr2=0x7fec8406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fec8406c920 0x7fec8406ede0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009a80 tx=0x7fec9c008040 comp rx=0 tx=0).stop 2024-04-26T19:51:32.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feca40b4c50 msgr2=0x7feca40d2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feca40b4c50 0x7feca40d2260 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fec94001f40 tx=0x7fec940014a0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 shutdown_connections 2024-04-26T19:51:32.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fec8406c920 0x7fec8406ede0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feca40b4c50 0x7feca40d2260 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feca4157680 0x7feca415ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 --2- 172.21.15.79:0/2736741283 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fec8c0051c0 0x7feca415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.424+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 >> 172.21.15.79:0/2736741283 conn(0x7feca40741a0 msgr2=0x7feca414f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:32.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.425+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 shutdown_connections 2024-04-26T19:51:32.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.425+0000 7feca8bf0640 1 -- 172.21.15.79:0/2736741283 wait complete. 2024-04-26T19:51:32.436 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T19:51:32.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.533+0000 7f676087e640 1 Processor -- start 2024-04-26T19:51:32.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.533+0000 7f676087e640 1 -- start start 2024-04-26T19:51:32.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.534+0000 7f676087e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f675c06ff00 0x7f675c148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.534+0000 7f676087e640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f675c0b1290 con 0x7f675c0d2cd0 2024-04-26T19:51:32.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.534+0000 7f676087e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f675c0d4090 con 0x7f675c06fb20 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.534+0000 7f676087e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f675c0d3140 con 0x7f675c06ff00 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.534+0000 7f675ad9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f675c0d2cd0 0x7f675c0aebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41552/0 (socket says 172.21.15.79:41552) 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.534+0000 7f675ad9d640 1 -- 172.21.15.79:0/1988160014 learned_addr learned my addr 172.21.15.79:0/1988160014 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.535+0000 7f6759d9b640 1 --2- 172.21.15.79:0/1988160014 >> v2:172.21.15.83:3300/0 conn(0x7f675c06ff00 0x7f675c148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.535+0000 7f675959a640 1 -- 172.21.15.79:0/1988160014 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3107587658 0 0) 0x7f675c0b1290 con 0x7f675c0d2cd0 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.535+0000 7f675959a640 1 -- 172.21.15.79:0/1988160014 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6744003520 con 0x7f675c0d2cd0 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.535+0000 7f675959a640 1 -- 172.21.15.79:0/1988160014 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 194009763 0 0) 0x7f6744003520 con 0x7f675c0d2cd0 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.535+0000 7f6759d9b640 1 -- 172.21.15.79:0/1988160014 >> v1:172.21.15.83:6789/0 conn(0x7f675c06fb20 legacy=0x7f675c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.535+0000 7f6759d9b640 1 -- 172.21.15.79:0/1988160014 >> v1:172.21.15.79:6789/0 conn(0x7f675c0d2cd0 legacy=0x7f675c0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.535+0000 7f6759d9b640 1 -- 172.21.15.79:0/1988160014 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c0d4370 con 0x7f675c06ff00 2024-04-26T19:51:32.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f6759d9b640 1 --2- 172.21.15.79:0/1988160014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06ff00 0x7f675c148fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f67500097a0 tx=0x7f675002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8e5c72fe75bde50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:32.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f675959a640 1 -- 172.21.15.79:0/1988160014 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750037070 con 0x7f675c06ff00 2024-04-26T19:51:32.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f675959a640 1 -- 172.21.15.79:0/1988160014 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f675002ace0 con 0x7f675c06ff00 2024-04-26T19:51:32.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f675959a640 1 -- 172.21.15.79:0/1988160014 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6754001670 con 0x7f675c06ff00 2024-04-26T19:51:32.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f676087e640 1 -- 172.21.15.79:0/1988160014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06ff00 msgr2=0x7f675c148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f676087e640 1 --2- 172.21.15.79:0/1988160014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06ff00 0x7f675c148fd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f67500097a0 tx=0x7f675002a250 comp rx=0 tx=0).stop 2024-04-26T19:51:32.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f676087e640 1 -- 172.21.15.79:0/1988160014 shutdown_connections 2024-04-26T19:51:32.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f676087e640 1 --2- 172.21.15.79:0/1988160014 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06ff00 0x7f675c148fd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f676087e640 1 -- 172.21.15.79:0/1988160014 >> 172.21.15.79:0/1988160014 conn(0x7f675c071c30 msgr2=0x7f675c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:32.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f676087e640 1 -- 172.21.15.79:0/1988160014 shutdown_connections 2024-04-26T19:51:32.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.536+0000 7f676087e640 1 -- 172.21.15.79:0/1988160014 wait complete. 2024-04-26T19:51:32.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.538+0000 7f676087e640 1 Processor -- start 2024-04-26T19:51:32.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.538+0000 7f676087e640 1 -- start start 2024-04-26T19:51:32.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.538+0000 7f676087e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06fb20 0x7f675c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.538+0000 7f676087e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f675c06ff00 0x7f675c0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.538+0000 7f676087e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f675c0d2cd0 0x7f675c0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.538+0000 7f676087e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f675c0d4c10 con 0x7f675c06ff00 2024-04-26T19:51:32.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.538+0000 7f676087e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f675c0d4930 con 0x7f675c06fb20 2024-04-26T19:51:32.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.538+0000 7f676087e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f675c0d3920 con 0x7f675c0d2cd0 2024-04-26T19:51:32.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f6759d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f675c06ff00 0x7f675c0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f6759d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f675c06ff00 0x7f675c0c50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35094/0 (socket says 172.21.15.79:35094) 2024-04-26T19:51:32.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f6759d9b640 1 -- 172.21.15.79:0/902200489 learned_addr learned my addr 172.21.15.79:0/902200489 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:32.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f675a59c640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06fb20 0x7f675c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f6759d9b640 1 -- 172.21.15.79:0/902200489 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f675c0d2cd0 msgr2=0x7f675c0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f675ad9d640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f675c0d2cd0 0x7f675c0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f6759d9b640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f675c0d2cd0 0x7f675c0c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f6759d9b640 1 -- 172.21.15.79:0/902200489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06fb20 msgr2=0x7f675c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f6759d9b640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06fb20 0x7f675c15df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f6759d9b640 1 -- 172.21.15.79:0/902200489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675c15e4b0 con 0x7f675c06ff00 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f675a59c640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06fb20 0x7f675c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f6759d9b640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f675c06ff00 0x7f675c0c50b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f67500098c0 tx=0x7f67500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f674affd640 1 -- 172.21.15.79:0/902200489 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750041020 con 0x7f675c06ff00 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f674affd640 1 -- 172.21.15.79:0/902200489 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6750003d40 con 0x7f675c06ff00 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675c134a60 con 0x7f675c06ff00 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f674affd640 1 -- 172.21.15.79:0/902200489 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6750037040 con 0x7f675c06ff00 2024-04-26T19:51:32.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.539+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f675c134fe0 con 0x7f675c06ff00 2024-04-26T19:51:32.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.540+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675c0506c0 con 0x7f675c06ff00 2024-04-26T19:51:32.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.541+0000 7f674affd640 1 -- 172.21.15.79:0/902200489 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6750044020 con 0x7f675c06ff00 2024-04-26T19:51:32.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.541+0000 7f674affd640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f673c06c8f0 0x7f673c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.541+0000 7f674affd640 1 -- 172.21.15.79:0/902200489 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f67500ac200 con 0x7f675c06ff00 2024-04-26T19:51:32.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.541+0000 7f675a59c640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f673c06c8f0 0x7f673c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.541+0000 7f675a59c640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f673c06c8f0 0x7f673c06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f674c0012e0 tx=0x7f674c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:32.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.545+0000 7f674affd640 1 -- 172.21.15.79:0/902200489 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f675004b070 con 0x7f675c06ff00 2024-04-26T19:51:32.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.758+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f675c06e590 con 0x7f675c06ff00 2024-04-26T19:51:32.759 INFO:tasks.ceph.mon.a.smithi079.stderr:2024-04-26T19:51:32.758+0000 7fbbc119c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T19:51:32.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.760+0000 7f674affd640 1 -- 172.21.15.79:0/902200489 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7f675c15e4b0 con 0x7f675c06ff00 2024-04-26T19:51:32.760 INFO:teuthology.orchestra.run.smithi079.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T19:51:32.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f673c06c8f0 msgr2=0x7f673c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f673c06c8f0 0x7f673c06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f674c0012e0 tx=0x7f674c009040 comp rx=0 tx=0).stop 2024-04-26T19:51:32.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f675c06ff00 msgr2=0x7f675c0c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f675c06ff00 0x7f675c0c50b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f67500098c0 tx=0x7f67500046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.763 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 shutdown_connections 2024-04-26T19:51:32.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f673c06c8f0 0x7f673c06edb0 secure :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f674c0012e0 tx=0x7f674c009040 comp rx=0 tx=0).stop 2024-04-26T19:51:32.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f675c0d2cd0 0x7f675c0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f675c06ff00 0x7f675c0c50b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 --2- 172.21.15.79:0/902200489 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f675c06fb20 0x7f675c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.762+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 >> 172.21.15.79:0/902200489 conn(0x7f675c071c30 msgr2=0x7f675c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:32.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.763+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 shutdown_connections 2024-04-26T19:51:32.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.763+0000 7f676087e640 1 -- 172.21.15.79:0/902200489 wait complete. 2024-04-26T19:51:32.766 INFO:tasks.ceph.mds.d.smithi083.stderr: -5428> 2024-04-26T19:50:35.072+0000 7fc38c99e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T19:51:32.774 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T19:51:32.798 INFO:tasks.ceph.mds.d.smithi083.stdout:starting mds.d at 2024-04-26T19:51:32.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.874+0000 7f766a21d640 1 Processor -- start 2024-04-26T19:51:32.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.874+0000 7f766a21d640 1 -- start start 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f766a21d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7664070030 0x7f766415d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f766a21d640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76640afe90 con 0x7f766415fba0 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f766a21d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7664160230 con 0x7f76640afab0 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f766a21d640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f7664160010 con 0x7f7664070030 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f7663fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7664070030 0x7f766415d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f7662ffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f766415fba0 0x7f766406f920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41560/0 (socket says 172.21.15.79:41560) 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f7663fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7664070030 0x7f766415d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35096/0 (socket says 172.21.15.79:35096) 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f7662ffd640 1 -- 172.21.15.79:0/1360116927 learned_addr learned my addr 172.21.15.79:0/1360116927 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f7663fff640 1 -- 172.21.15.79:0/1360116927 >> v1:172.21.15.83:6789/0 conn(0x7f76640afab0 legacy=0x7f7664141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f7663fff640 1 -- 172.21.15.79:0/1360116927 >> v1:172.21.15.79:6789/0 conn(0x7f766415fba0 legacy=0x7f766406f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f7663fff640 1 -- 172.21.15.79:0/1360116927 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7664160510 con 0x7f7664070030 2024-04-26T19:51:32.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.875+0000 7f76627fc640 1 -- 172.21.15.79:0/1360116927 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2753999839 0 0) 0x7f76640afe90 con 0x7f766415fba0 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.876+0000 7f7663fff640 1 --2- 172.21.15.79:0/1360116927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664070030 0x7f766415d030 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f765c009d90 tx=0x7f765c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37910bffc912344c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.876+0000 7f76627fc640 1 -- 172.21.15.79:0/1360116927 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c007740 con 0x7f7664070030 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.876+0000 7f76627fc640 1 -- 172.21.15.79:0/1360116927 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f765c0078e0 con 0x7f7664070030 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.876+0000 7f76627fc640 1 -- 172.21.15.79:0/1360116927 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c007cb0 con 0x7f7664070030 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.876+0000 7f766a21d640 1 -- 172.21.15.79:0/1360116927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664070030 msgr2=0x7f766415d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.876+0000 7f766a21d640 1 --2- 172.21.15.79:0/1360116927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664070030 0x7f766415d030 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f765c009d90 tx=0x7f765c02a6a0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.877+0000 7f766a21d640 1 -- 172.21.15.79:0/1360116927 shutdown_connections 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.877+0000 7f766a21d640 1 --2- 172.21.15.79:0/1360116927 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7664070030 0x7f766415d030 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.877+0000 7f766a21d640 1 -- 172.21.15.79:0/1360116927 >> 172.21.15.79:0/1360116927 conn(0x7f76640b2670 msgr2=0x7f76640b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:32.878 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.877+0000 7f766a21d640 1 -- 172.21.15.79:0/1360116927 shutdown_connections 2024-04-26T19:51:32.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.877+0000 7f766a21d640 1 -- 172.21.15.79:0/1360116927 wait complete. 2024-04-26T19:51:32.879 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.878+0000 7f766a21d640 1 Processor -- start 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.878+0000 7f766a21d640 1 -- start start 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f766a21d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7664070030 0x7f766413abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f766a21d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76640afab0 0x7f766415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f766a21d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766415fba0 0x7f766415e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f766a21d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7664142480 con 0x7f766415fba0 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f766a21d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7664141850 con 0x7f76640afab0 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f766a21d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f7664142600 con 0x7f7664070030 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7663fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766415fba0 0x7f766415e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7663fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766415fba0 0x7f766415e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35110/0 (socket says 172.21.15.79:35110) 2024-04-26T19:51:32.881 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7663fff640 1 -- 172.21.15.79:0/628569044 learned_addr learned my addr 172.21.15.79:0/628569044 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7662ffd640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76640afab0 0x7f766415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f76637fe640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7664070030 0x7f766413abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7663fff640 1 -- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7664070030 msgr2=0x7f766413abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7663fff640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7664070030 0x7f766413abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7663fff640 1 -- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76640afab0 msgr2=0x7f766415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7663fff640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76640afab0 0x7f766415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7663fff640 1 -- 172.21.15.79:0/628569044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76641562f0 con 0x7f766415fba0 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7662ffd640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76640afab0 0x7f766415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f76637fe640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7664070030 0x7f766413abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f7663fff640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766415fba0 0x7f766415e2e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f765c009eb0 tx=0x7f765c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f764bfff640 1 -- 172.21.15.79:0/628569044 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c040070 con 0x7f766415fba0 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f764bfff640 1 -- 172.21.15.79:0/628569044 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f765c031c90 con 0x7f766415fba0 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7664157940 con 0x7f766415fba0 2024-04-26T19:51:32.882 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f764bfff640 1 -- 172.21.15.79:0/628569044 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765c045680 con 0x7f766415fba0 2024-04-26T19:51:32.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.879+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7664157f00 con 0x7f766415fba0 2024-04-26T19:51:32.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.880+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76640506c0 con 0x7f766415fba0 2024-04-26T19:51:32.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.881+0000 7f764bfff640 1 -- 172.21.15.79:0/628569044 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f765c039040 con 0x7f766415fba0 2024-04-26T19:51:32.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.881+0000 7f764bfff640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f765006c8f0 0x7f765006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:32.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.881+0000 7f764bfff640 1 -- 172.21.15.79:0/628569044 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f765c0ad910 con 0x7f766415fba0 2024-04-26T19:51:32.883 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.881+0000 7f76637fe640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f765006c8f0 0x7f765006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:32.884 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.881+0000 7f76637fe640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f765006c8f0 0x7f765006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7658003eb0 tx=0x7f7658009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:32.886 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:32.885+0000 7f764bfff640 1 -- 172.21.15.79:0/628569044 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f765c07ba30 con 0x7f766415fba0 2024-04-26T19:51:33.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.097+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f76641530b0 con 0x7f766415fba0 2024-04-26T19:51:33.764 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.764+0000 7f764bfff640 1 -- 172.21.15.79:0/628569044 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f765c07b330 con 0x7f766415fba0 2024-04-26T19:51:33.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.766+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f765006c8f0 msgr2=0x7f765006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.766+0000 7f766a21d640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f765006c8f0 0x7f765006edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7658003eb0 tx=0x7f7658009040 comp rx=0 tx=0).stop 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.766+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766415fba0 msgr2=0x7f766415e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.766+0000 7f766a21d640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766415fba0 0x7f766415e2e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f765c009eb0 tx=0x7f765c004a40 comp rx=0 tx=0).stop 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.766+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 shutdown_connections 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.766+0000 7f766a21d640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f765006c8f0 0x7f765006edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.766+0000 7f766a21d640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f766415fba0 0x7f766415e2e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.766+0000 7f766a21d640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f76640afab0 0x7f766415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.766+0000 7f766a21d640 1 --2- 172.21.15.79:0/628569044 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7664070030 0x7f766413abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.767+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 >> 172.21.15.79:0/628569044 conn(0x7f76640b2670 msgr2=0x7f766406f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.767+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 shutdown_connections 2024-04-26T19:51:33.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.767+0000 7f766a21d640 1 -- 172.21.15.79:0/628569044 wait complete. 2024-04-26T19:51:33.781 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T19:51:33.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.886+0000 7fa2033c1640 1 Processor -- start 2024-04-26T19:51:33.887 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.887+0000 7fa2033c1640 1 -- start start 2024-04-26T19:51:33.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.887+0000 7fa2033c1640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa1fc06df80 0x7fa1fc06f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:33.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.887+0000 7fa2033c1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa1fc144ca0 0x7fa1fc1450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:33.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.887+0000 7fa2033c1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1fc145620 con 0x7fa1fc159b30 2024-04-26T19:51:33.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.887+0000 7fa2033c1640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fa1fc145bb0 con 0x7fa1fc06df80 2024-04-26T19:51:33.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.887+0000 7fa2033c1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa1fc06f090 con 0x7fa1fc144ca0 2024-04-26T19:51:33.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa20115d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa1fc159b30 0x7fa1fc160560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41576/0 (socket says 172.21.15.79:41576) 2024-04-26T19:51:33.890 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa20095c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa1fc06df80 0x7fa1fc06f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa20115d640 1 -- 172.21.15.79:0/2266884785 learned_addr learned my addr 172.21.15.79:0/2266884785 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa20195e640 1 --2- 172.21.15.79:0/2266884785 >> v2:172.21.15.83:3300/0 conn(0x7fa1fc144ca0 0x7fa1fc1450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa1f3fff640 1 -- 172.21.15.79:0/2266884785 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2929674243 0 0) 0x7fa1fc145620 con 0x7fa1fc159b30 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa1f3fff640 1 -- 172.21.15.79:0/2266884785 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1e8003520 con 0x7fa1fc159b30 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa20095c640 1 -- 172.21.15.79:0/2266884785 >> v2:172.21.15.83:3300/0 conn(0x7fa1fc144ca0 msgr2=0x7fa1fc1450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa20095c640 1 --2- 172.21.15.79:0/2266884785 >> v2:172.21.15.83:3300/0 conn(0x7fa1fc144ca0 0x7fa1fc1450e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa20095c640 1 -- 172.21.15.79:0/2266884785 >> v1:172.21.15.79:6789/0 conn(0x7fa1fc159b30 legacy=0x7fa1fc160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.888+0000 7fa20095c640 1 -- 172.21.15.79:0/2266884785 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc06e480 con 0x7fa1fc06df80 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa20095c640 1 --2- 172.21.15.79:0/2266884785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 0x7fa1fc06f520 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc06fcf0 tx=0x7fa1ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e8759fa7633b74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa1f3fff640 1 -- 172.21.15.79:0/2266884785 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec037070 con 0x7fa1fc06df80 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa1f3fff640 1 -- 172.21.15.79:0/2266884785 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1ec032440 con 0x7fa1fc06df80 2024-04-26T19:51:33.891 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa1f3fff640 1 -- 172.21.15.79:0/2266884785 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1ec0327f0 con 0x7fa1fc06df80 2024-04-26T19:51:33.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa2033c1640 1 -- 172.21.15.79:0/2266884785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 msgr2=0x7fa1fc06f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:33.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa2033c1640 1 --2- 172.21.15.79:0/2266884785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 0x7fa1fc06f520 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc06fcf0 tx=0x7fa1ec02a1a0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa2033c1640 1 -- 172.21.15.79:0/2266884785 shutdown_connections 2024-04-26T19:51:33.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa2033c1640 1 --2- 172.21.15.79:0/2266884785 >> v2:172.21.15.83:3300/0 conn(0x7fa1fc144ca0 0x7fa1fc1450e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa2033c1640 1 --2- 172.21.15.79:0/2266884785 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 0x7fa1fc06f520 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa2033c1640 1 -- 172.21.15.79:0/2266884785 >> 172.21.15.79:0/2266884785 conn(0x7fa1fc0685d0 msgr2=0x7fa1fc0d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:33.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa2033c1640 1 -- 172.21.15.79:0/2266884785 shutdown_connections 2024-04-26T19:51:33.892 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.889+0000 7fa2033c1640 1 -- 172.21.15.79:0/2266884785 wait complete. 2024-04-26T19:51:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.890+0000 7fa2033c1640 1 Processor -- start 2024-04-26T19:51:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa2033c1640 1 -- start start 2024-04-26T19:51:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa2033c1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 0x7fa1fc1f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa2033c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc144ca0 0x7fa1fc1f57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa2033c1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1fc159b30 0x7fa1fc0721f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa2033c1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc06ed20 con 0x7fa1fc06df80 2024-04-26T19:51:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa2033c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc06ea40 con 0x7fa1fc144ca0 2024-04-26T19:51:33.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa2033c1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa1fc159f10 con 0x7fa1fc159b30 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa20115d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 0x7fa1fc1f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa20115d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 0x7fa1fc1f5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35128/0 (socket says 172.21.15.79:35128) 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.891+0000 7fa20115d640 1 -- 172.21.15.79:0/1750868537 learned_addr learned my addr 172.21.15.79:0/1750868537 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa20195e640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1fc159b30 0x7fa1fc0721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa20115d640 1 -- 172.21.15.79:0/1750868537 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1fc159b30 msgr2=0x7fa1fc0721f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa20095c640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc144ca0 0x7fa1fc1f57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa20115d640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1fc159b30 0x7fa1fc0721f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa20115d640 1 -- 172.21.15.79:0/1750868537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc144ca0 msgr2=0x7fa1fc1f57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa20115d640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc144ca0 0x7fa1fc1f57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa20115d640 1 -- 172.21.15.79:0/1750868537 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc162540 con 0x7fa1fc06df80 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa20115d640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 0x7fa1fc1f5ef0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c430 tx=0x7fa1f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa1f1ffb640 1 -- 172.21.15.79:0/1750868537 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8017070 con 0x7fa1fc06df80 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa1f1ffb640 1 -- 172.21.15.79:0/1750868537 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1f8005ce0 con 0x7fa1fc06df80 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa1f1ffb640 1 -- 172.21.15.79:0/1750868537 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f80072f0 con 0x7fa1fc06df80 2024-04-26T19:51:33.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc160e80 con 0x7fa1fc06df80 2024-04-26T19:51:33.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.892+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc161440 con 0x7fa1fc06df80 2024-04-26T19:51:33.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.893+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1fc0506c0 con 0x7fa1fc06df80 2024-04-26T19:51:33.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.893+0000 7fa1f1ffb640 1 -- 172.21.15.79:0/1750868537 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa1f801a070 con 0x7fa1fc06df80 2024-04-26T19:51:33.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.894+0000 7fa1f1ffb640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1e006c8f0 0x7fa1e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:33.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.894+0000 7fa1f1ffb640 1 -- 172.21.15.79:0/1750868537 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fa1f808b470 con 0x7fa1fc06df80 2024-04-26T19:51:33.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.894+0000 7fa20095c640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1e006c8f0 0x7fa1e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:33.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.894+0000 7fa20095c640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1e006c8f0 0x7fa1e006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec034000 tx=0x7fa1ec02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:33.899 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:33.898+0000 7fa1f1ffb640 1 -- 172.21.15.79:0/1750868537 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa1fc162540 con 0x7fa1fc06df80 2024-04-26T19:51:34.072 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.072+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa1fc0732b0 con 0x7fa1fc06df80 2024-04-26T19:51:34.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.772+0000 7fa1f1ffb640 1 -- 172.21.15.79:0/1750868537 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7fa1f8059440 con 0x7fa1fc06df80 2024-04-26T19:51:34.773 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' removed 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.774+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1e006c8f0 msgr2=0x7fa1e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.774+0000 7fa2033c1640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1e006c8f0 0x7fa1e006edb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec034000 tx=0x7fa1ec02e000 comp rx=0 tx=0).stop 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.774+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 msgr2=0x7fa1fc1f5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.774+0000 7fa2033c1640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 0x7fa1fc1f5ef0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa1f800c430 tx=0x7fa1f800c8f0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.775+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 shutdown_connections 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.775+0000 7fa2033c1640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa1e006c8f0 0x7fa1e006edb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.775+0000 7fa2033c1640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa1fc159b30 0x7fa1fc0721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.775+0000 7fa2033c1640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa1fc144ca0 0x7fa1fc1f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.775+0000 7fa2033c1640 1 --2- 172.21.15.79:0/1750868537 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa1fc06df80 0x7fa1fc1f5ef0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.775+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 >> 172.21.15.79:0/1750868537 conn(0x7fa1fc0685d0 msgr2=0x7fa1fc1450e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:34.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.775+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 shutdown_connections 2024-04-26T19:51:34.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.775+0000 7fa2033c1640 1 -- 172.21.15.79:0/1750868537 wait complete. 2024-04-26T19:51:34.790 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T19:51:34.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.895+0000 7fbc33791640 1 Processor -- start 2024-04-26T19:51:34.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.895+0000 7fbc33791640 1 -- start start 2024-04-26T19:51:34.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc33791640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c160650 0x7fbc2c161880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:34.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc33791640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbc2c06eeb0 0x7fbc2c06f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:34.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc33791640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc2c0cfea0 0x7fbc2c0d0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc33791640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fbc2c161dc0 con 0x7fbc2c06eeb0 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc33791640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbc2c06f930 con 0x7fbc2c0cfea0 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc33791640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c06fc10 con 0x7fbc2c160650 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc30d2c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbc2c06eeb0 0x7fbc2c06f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc30d2c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbc2c06eeb0 0x7fbc2c06f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35136/0 (socket says 172.21.15.79:35136) 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc30d2c640 1 -- 172.21.15.79:0/2478338850 learned_addr learned my addr 172.21.15.79:0/2478338850 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc31d2e640 1 --2- 172.21.15.79:0/2478338850 >> v2:172.21.15.83:3300/0 conn(0x7fbc2c0cfea0 0x7fbc2c0d0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.896+0000 7fbc3152d640 1 --2- 172.21.15.79:0/2478338850 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c160650 0x7fbc2c161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.897+0000 7fbc30d2c640 1 -- 172.21.15.79:0/2478338850 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c160650 msgr2=0x7fbc2c161880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.897+0000 7fbc30d2c640 1 --2- 172.21.15.79:0/2478338850 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c160650 0x7fbc2c161880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.897+0000 7fbc30d2c640 1 -- 172.21.15.79:0/2478338850 >> v2:172.21.15.83:3300/0 conn(0x7fbc2c0cfea0 msgr2=0x7fbc2c0d0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.897+0000 7fbc30d2c640 1 --2- 172.21.15.79:0/2478338850 >> v2:172.21.15.83:3300/0 conn(0x7fbc2c0cfea0 0x7fbc2c0d0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.897+0000 7fbc30d2c640 1 -- 172.21.15.79:0/2478338850 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c161080 con 0x7fbc2c06eeb0 2024-04-26T19:51:34.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.897+0000 7fbc31d2e640 1 --2- 172.21.15.79:0/2478338850 >> v2:172.21.15.83:3300/0 conn(0x7fbc2c0cfea0 0x7fbc2c0d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:34.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.897+0000 7fbc30d2c640 1 --2- 172.21.15.79:0/2478338850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c06eeb0 0x7fbc2c06f310 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c070e30 tx=0x7fbc1c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6d35bb0413febb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:34.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.897+0000 7fbc23fff640 1 -- 172.21.15.79:0/2478338850 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c037070 con 0x7fbc2c06eeb0 2024-04-26T19:51:34.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc23fff640 1 -- 172.21.15.79:0/2478338850 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1c032440 con 0x7fbc2c06eeb0 2024-04-26T19:51:34.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc23fff640 1 -- 172.21.15.79:0/2478338850 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c032810 con 0x7fbc2c06eeb0 2024-04-26T19:51:34.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc33791640 1 -- 172.21.15.79:0/2478338850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c06eeb0 msgr2=0x7fbc2c06f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:34.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc33791640 1 --2- 172.21.15.79:0/2478338850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c06eeb0 0x7fbc2c06f310 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c070e30 tx=0x7fbc1c02a1e0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc33791640 1 -- 172.21.15.79:0/2478338850 shutdown_connections 2024-04-26T19:51:34.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc33791640 1 --2- 172.21.15.79:0/2478338850 >> v2:172.21.15.83:3300/0 conn(0x7fbc2c0cfea0 0x7fbc2c0d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc33791640 1 --2- 172.21.15.79:0/2478338850 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c06eeb0 0x7fbc2c06f310 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc33791640 1 --2- 172.21.15.79:0/2478338850 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c160650 0x7fbc2c161880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc33791640 1 -- 172.21.15.79:0/2478338850 >> 172.21.15.79:0/2478338850 conn(0x7fbc2c0b3f00 msgr2=0x7fbc2c0b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:34.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc33791640 1 -- 172.21.15.79:0/2478338850 shutdown_connections 2024-04-26T19:51:34.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.898+0000 7fbc33791640 1 -- 172.21.15.79:0/2478338850 wait complete. 2024-04-26T19:51:34.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.899+0000 7fbc33791640 1 Processor -- start 2024-04-26T19:51:34.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.899+0000 7fbc33791640 1 -- start start 2024-04-26T19:51:34.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc33791640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc2c06eeb0 0x7fbc2c15b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:34.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc33791640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c0cfea0 0x7fbc2c138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:34.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc33791640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c160650 0x7fbc2c139660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:34.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc33791640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c070760 con 0x7fbc2c160650 2024-04-26T19:51:34.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc33791640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2c0705e0 con 0x7fbc2c06eeb0 2024-04-26T19:51:34.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc33791640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2c160a30 con 0x7fbc2c0cfea0 2024-04-26T19:51:34.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc31d2e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c160650 0x7fbc2c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:34.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc31d2e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c160650 0x7fbc2c139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35150/0 (socket says 172.21.15.79:35150) 2024-04-26T19:51:34.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc31d2e640 1 -- 172.21.15.79:0/1624317514 learned_addr learned my addr 172.21.15.79:0/1624317514 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc30d2c640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c0cfea0 0x7fbc2c138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc3152d640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc2c06eeb0 0x7fbc2c15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc31d2e640 1 -- 172.21.15.79:0/1624317514 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c0cfea0 msgr2=0x7fbc2c138ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc31d2e640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c0cfea0 0x7fbc2c138ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc31d2e640 1 -- 172.21.15.79:0/1624317514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc2c06eeb0 msgr2=0x7fbc2c15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc31d2e640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc2c06eeb0 0x7fbc2c15b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc31d2e640 1 -- 172.21.15.79:0/1624317514 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2c1f30d0 con 0x7fbc2c160650 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.900+0000 7fbc3152d640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc2c06eeb0 0x7fbc2c15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.901+0000 7fbc31d2e640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c160650 0x7fbc2c139660 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400e610 tx=0x7fbc2400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.901+0000 7fbc21ffb640 1 -- 172.21.15.79:0/1624317514 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400f040 con 0x7fbc2c160650 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.901+0000 7fbc21ffb640 1 -- 172.21.15.79:0/1624317514 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc24004230 con 0x7fbc2c160650 2024-04-26T19:51:34.904 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.901+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc2c1f33b0 con 0x7fbc2c160650 2024-04-26T19:51:34.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.901+0000 7fbc21ffb640 1 -- 172.21.15.79:0/1624317514 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24003610 con 0x7fbc2c160650 2024-04-26T19:51:34.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.901+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc2c1f3970 con 0x7fbc2c160650 2024-04-26T19:51:34.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.902+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2c0506c0 con 0x7fbc2c160650 2024-04-26T19:51:34.906 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.903+0000 7fbc21ffb640 1 -- 172.21.15.79:0/1624317514 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc240037b0 con 0x7fbc2c160650 2024-04-26T19:51:34.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.903+0000 7fbc21ffb640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbc1006c8f0 0x7fbc1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:34.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.903+0000 7fbc21ffb640 1 -- 172.21.15.79:0/1624317514 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7fbc2408ba20 con 0x7fbc2c160650 2024-04-26T19:51:34.908 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.903+0000 7fbc3152d640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbc1006c8f0 0x7fbc1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:34.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.903+0000 7fbc3152d640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbc1006c8f0 0x7fbc1006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c142f20 tx=0x7fbc280039f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:34.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:34.907+0000 7fbc21ffb640 1 -- 172.21.15.79:0/1624317514 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbc24017020 con 0x7fbc2c160650 2024-04-26T19:51:35.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.089+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbc2c06ffc0 con 0x7fbc2c160650 2024-04-26T19:51:35.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.773+0000 7fbc21ffb640 1 -- 172.21.15.79:0/1624317514 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7fbc24059560 con 0x7fbc2c160650 2024-04-26T19:51:35.774 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' removed 2024-04-26T19:51:35.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.775+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbc1006c8f0 msgr2=0x7fbc1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:35.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.775+0000 7fbc33791640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbc1006c8f0 0x7fbc1006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c142f20 tx=0x7fbc280039f0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.775+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c160650 msgr2=0x7fbc2c139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:35.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.775+0000 7fbc33791640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c160650 0x7fbc2c139660 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400e610 tx=0x7fbc2400ead0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.775+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 shutdown_connections 2024-04-26T19:51:35.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.775+0000 7fbc33791640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbc1006c8f0 0x7fbc1006edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.775+0000 7fbc33791640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc2c160650 0x7fbc2c139660 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.775+0000 7fbc33791640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc2c0cfea0 0x7fbc2c138ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.776+0000 7fbc33791640 1 --2- 172.21.15.79:0/1624317514 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc2c06eeb0 0x7fbc2c15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.776+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 >> 172.21.15.79:0/1624317514 conn(0x7fbc2c0b3f00 msgr2=0x7fbc2c15c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:35.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.776+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 shutdown_connections 2024-04-26T19:51:35.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.776+0000 7fbc33791640 1 -- 172.21.15.79:0/1624317514 wait complete. 2024-04-26T19:51:35.792 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-26T19:51:35.893 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.893+0000 7f1b99b7f640 1 Processor -- start 2024-04-26T19:51:35.894 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.893+0000 7f1b99b7f640 1 -- start start 2024-04-26T19:51:35.895 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b99b7f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1b941422d0 0x7f1b9406f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b99b7f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b940718a0 0x7f1b940721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b99b7f640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b9406fb60 con 0x7f1b94072710 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b99b7f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1b9406fe40 con 0x7f1b941422d0 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b99b7f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1b94070120 con 0x7f1b940718a0 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b93fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f1b94072710 0x7f1b94162fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41592/0 (socket says 172.21.15.79:41592) 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b93fff640 1 -- 172.21.15.79:0/1473687124 learned_addr learned my addr 172.21.15.79:0/1473687124 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b937fe640 1 --2- 172.21.15.79:0/1473687124 >> v2:172.21.15.83:3300/0 conn(0x7f1b941422d0 0x7f1b9406f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b92ffd640 1 --2- 172.21.15.79:0/1473687124 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b940718a0 0x7f1b940721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b927fc640 1 -- 172.21.15.79:0/1473687124 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3104046439 0 0) 0x7f1b9406fb60 con 0x7f1b94072710 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b927fc640 1 -- 172.21.15.79:0/1473687124 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b7c003520 con 0x7f1b94072710 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b92ffd640 1 -- 172.21.15.79:0/1473687124 >> v2:172.21.15.83:3300/0 conn(0x7f1b941422d0 msgr2=0x7f1b9406f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b92ffd640 1 --2- 172.21.15.79:0/1473687124 >> v2:172.21.15.83:3300/0 conn(0x7f1b941422d0 0x7f1b9406f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b92ffd640 1 -- 172.21.15.79:0/1473687124 >> v1:172.21.15.79:6789/0 conn(0x7f1b94072710 legacy=0x7f1b94162fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:35.896 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b927fc640 1 -- 172.21.15.79:0/1473687124 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2191201125 0 0) 0x7f1b7c003520 con 0x7f1b94072710 2024-04-26T19:51:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b92ffd640 1 -- 172.21.15.79:0/1473687124 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b94070400 con 0x7f1b940718a0 2024-04-26T19:51:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.894+0000 7f1b937fe640 1 --2- 172.21.15.79:0/1473687124 >> v2:172.21.15.83:3300/0 conn(0x7f1b941422d0 0x7f1b9406f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.895+0000 7f1b92ffd640 1 --2- 172.21.15.79:0/1473687124 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b940718a0 0x7f1b940721d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b880097a0 tx=0x7f1b8802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cae0065b4805c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.895+0000 7f1b927fc640 1 -- 172.21.15.79:0/1473687124 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b88037070 con 0x7f1b940718a0 2024-04-26T19:51:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.895+0000 7f1b927fc640 1 -- 172.21.15.79:0/1473687124 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8802aba0 con 0x7f1b940718a0 2024-04-26T19:51:35.897 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.895+0000 7f1b927fc640 1 -- 172.21.15.79:0/1473687124 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c0019f0 con 0x7f1b940718a0 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.896+0000 7f1b99b7f640 1 -- 172.21.15.79:0/1473687124 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b940718a0 msgr2=0x7f1b940721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.896+0000 7f1b99b7f640 1 --2- 172.21.15.79:0/1473687124 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b940718a0 0x7f1b940721d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b880097a0 tx=0x7f1b8802a110 comp rx=0 tx=0).stop 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.896+0000 7f1b99b7f640 1 -- 172.21.15.79:0/1473687124 shutdown_connections 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.896+0000 7f1b99b7f640 1 --2- 172.21.15.79:0/1473687124 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b940718a0 0x7f1b940721d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.896+0000 7f1b99b7f640 1 --2- 172.21.15.79:0/1473687124 >> v2:172.21.15.83:3300/0 conn(0x7f1b941422d0 0x7f1b9406f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.896+0000 7f1b99b7f640 1 -- 172.21.15.79:0/1473687124 >> 172.21.15.79:0/1473687124 conn(0x7f1b940b31c0 msgr2=0x7f1b940b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.896+0000 7f1b99b7f640 1 -- 172.21.15.79:0/1473687124 shutdown_connections 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.896+0000 7f1b99b7f640 1 -- 172.21.15.79:0/1473687124 wait complete. 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.897+0000 7f1b99b7f640 1 Processor -- start 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.897+0000 7f1b99b7f640 1 -- start start 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b99b7f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b940718a0 0x7f1b941f52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:35.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b99b7f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94072710 0x7f1b941f8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b99b7f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b941422d0 0x7f1b94162340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b99b7f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1b94142830 con 0x7f1b940718a0 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b99b7f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1b941426b0 con 0x7f1b94072710 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b99b7f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9406f730 con 0x7f1b941422d0 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b937fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b940718a0 0x7f1b941f52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b937fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b940718a0 0x7f1b941f52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35166/0 (socket says 172.21.15.79:35166) 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b937fe640 1 -- 172.21.15.79:0/3297874519 learned_addr learned my addr 172.21.15.79:0/3297874519 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b93fff640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b941422d0 0x7f1b94162340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b92ffd640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94072710 0x7f1b941f8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b937fe640 1 -- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b941422d0 msgr2=0x7f1b94162340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b937fe640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b941422d0 0x7f1b94162340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b937fe640 1 -- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94072710 msgr2=0x7f1b941f8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b937fe640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94072710 0x7f1b941f8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:35.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b937fe640 1 -- 172.21.15.79:0/3297874519 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b940cf240 con 0x7f1b940718a0 2024-04-26T19:51:35.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b93fff640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b941422d0 0x7f1b94162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:35.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b92ffd640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94072710 0x7f1b941f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:35.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.898+0000 7f1b937fe640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b940718a0 0x7f1b941f52d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400ebe0 tx=0x7f1b8400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:35.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.899+0000 7f1b7bfff640 1 -- 172.21.15.79:0/3297874519 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84017070 con 0x7f1b940718a0 2024-04-26T19:51:35.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.899+0000 7f1b7bfff640 1 -- 172.21.15.79:0/3297874519 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b8400cc30 con 0x7f1b940718a0 2024-04-26T19:51:35.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.899+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b940cf520 con 0x7f1b940718a0 2024-04-26T19:51:35.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.899+0000 7f1b7bfff640 1 -- 172.21.15.79:0/3297874519 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b84013470 con 0x7f1b940718a0 2024-04-26T19:51:35.902 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.899+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b941f8580 con 0x7f1b940718a0 2024-04-26T19:51:35.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.900+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b940506c0 con 0x7f1b940718a0 2024-04-26T19:51:35.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.900+0000 7f1b7bfff640 1 -- 172.21.15.79:0/3297874519 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b8401a070 con 0x7f1b940718a0 2024-04-26T19:51:35.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.900+0000 7f1b7bfff640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1b8006c8f0 0x7f1b8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:35.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.901+0000 7f1b7bfff640 1 -- 172.21.15.79:0/3297874519 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f1b8408bf60 con 0x7f1b940718a0 2024-04-26T19:51:35.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.901+0000 7f1b92ffd640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1b8006c8f0 0x7f1b8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:35.903 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.901+0000 7f1b92ffd640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1b8006c8f0 0x7f1b8006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1b88034000 tx=0x7f1b8802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:35.905 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:35.905+0000 7f1b7bfff640 1 -- 172.21.15.79:0/3297874519 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1b8405a280 con 0x7f1b940718a0 2024-04-26T19:51:36.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.088+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1b940afcf0 con 0x7f1b940718a0 2024-04-26T19:51:36.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.784+0000 7f1b7bfff640 1 -- 172.21.15.79:0/3297874519 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f1b84059b80 con 0x7f1b940718a0 2024-04-26T19:51:36.785 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data_ec' removed 2024-04-26T19:51:36.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1b8006c8f0 msgr2=0x7f1b8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:36.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1b8006c8f0 0x7f1b8006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1b88034000 tx=0x7f1b8802e000 comp rx=0 tx=0).stop 2024-04-26T19:51:36.789 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b940718a0 msgr2=0x7f1b941f52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:36.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b940718a0 0x7f1b941f52d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400ebe0 tx=0x7f1b8400c520 comp rx=0 tx=0).stop 2024-04-26T19:51:36.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 shutdown_connections 2024-04-26T19:51:36.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f1b8006c8f0 0x7f1b8006edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:36.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1b941422d0 0x7f1b94162340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:36.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1b94072710 0x7f1b941f8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:36.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 --2- 172.21.15.79:0/3297874519 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1b940718a0 0x7f1b941f52d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:36.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.788+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 >> 172.21.15.79:0/3297874519 conn(0x7f1b940b31c0 msgr2=0x7f1b940b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:36.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.789+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 shutdown_connections 2024-04-26T19:51:36.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.789+0000 7f1b99b7f640 1 -- 172.21.15.79:0/3297874519 wait complete. 2024-04-26T19:51:36.804 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-26T19:51:36.804 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-26T19:51:36.804 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T19:51:36.804 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:51:36.805 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-26T19:51:36.805 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-26T19:51:36.805 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T19:51:36.805 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:51:36.805 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-26T19:51:36.805 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-26T19:51:36.805 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T19:51:36.806 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:51:36.806 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-26T19:51:36.806 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-26T19:51:36.806 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T19:51:36.807 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:51:36.815 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T19:51:36.815 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-26T19:51:36.816 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T19:51:36.816 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-26T19:51:36.817 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T19:51:36.817 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-26T19:51:36.819 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T19:51:36.819 DEBUG:teuthology.orchestra.run.smithi083:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-26T19:51:36.858 INFO:tasks.ceph.mds.a:Started 2024-04-26T19:51:36.858 DEBUG:teuthology.parallel:result is None 2024-04-26T19:51:36.859 INFO:tasks.ceph.mds.c:Started 2024-04-26T19:51:36.859 DEBUG:teuthology.parallel:result is None 2024-04-26T19:51:36.861 INFO:tasks.ceph.mds.b:Started 2024-04-26T19:51:36.861 DEBUG:teuthology.parallel:result is None 2024-04-26T19:51:36.863 INFO:tasks.ceph.mds.d:Started 2024-04-26T19:51:36.863 DEBUG:teuthology.parallel:result is None 2024-04-26T19:51:36.863 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-26T19:51:36.940 INFO:tasks.ceph.mds.c.smithi079.stdout:starting mds.c at 2024-04-26T19:51:36.940 INFO:tasks.ceph.mds.a.smithi079.stdout:starting mds.a at 2024-04-26T19:51:36.943 INFO:tasks.ceph.mds.b.smithi083.stdout:starting mds.b at 2024-04-26T19:51:36.943 INFO:tasks.ceph.mds.d.smithi083.stdout:starting mds.d at 2024-04-26T19:51:36.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.971+0000 7f358c864640 1 Processor -- start 2024-04-26T19:51:36.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.971+0000 7f358c864640 1 -- start start 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f358c864640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f358806ff00 0x7f3588148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f358c864640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35880b1290 con 0x7f35880d2cd0 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f358c864640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35880d4090 con 0x7f358806fb20 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f358c864640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f35880d3140 con 0x7f358806ff00 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f3586d9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f35880d2cd0 0x7f35880aebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:41606/0 (socket says 172.21.15.79:41606) 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f3586d9d640 1 -- 172.21.15.79:0/2338278764 learned_addr learned my addr 172.21.15.79:0/2338278764 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f3585d9b640 1 --2- 172.21.15.79:0/2338278764 >> v2:172.21.15.83:3300/0 conn(0x7f358806ff00 0x7f3588148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f358559a640 1 -- 172.21.15.79:0/2338278764 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1591374981 0 0) 0x7f35880b1290 con 0x7f35880d2cd0 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f358559a640 1 -- 172.21.15.79:0/2338278764 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3570003520 con 0x7f35880d2cd0 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f358559a640 1 -- 172.21.15.79:0/2338278764 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3964560419 0 0) 0x7f35880d4090 con 0x7f358806fb20 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f358559a640 1 -- 172.21.15.79:0/2338278764 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35880b1290 con 0x7f358806fb20 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f3585d9b640 1 -- 172.21.15.79:0/2338278764 >> v1:172.21.15.83:6789/0 conn(0x7f358806fb20 legacy=0x7f35881488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f358559a640 1 -- 172.21.15.79:0/2338278764 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1707090948 0 0) 0x7f3570003520 con 0x7f35880d2cd0 2024-04-26T19:51:36.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f3585d9b640 1 -- 172.21.15.79:0/2338278764 >> v1:172.21.15.79:6789/0 conn(0x7f35880d2cd0 legacy=0x7f35880aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:36.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.972+0000 7f3585d9b640 1 -- 172.21.15.79:0/2338278764 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35880d4370 con 0x7f358806ff00 2024-04-26T19:51:36.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.973+0000 7f3585d9b640 1 --2- 172.21.15.79:0/2338278764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 0x7f3588148fd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f357c0097a0 tx=0x7f357c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b099d02fc20abb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:36.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.973+0000 7f358559a640 1 -- 172.21.15.79:0/2338278764 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c037070 con 0x7f358806ff00 2024-04-26T19:51:36.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.973+0000 7f358559a640 1 -- 172.21.15.79:0/2338278764 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f357c02ace0 con 0x7f358806ff00 2024-04-26T19:51:36.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.973+0000 7f358559a640 1 -- 172.21.15.79:0/2338278764 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800019f0 con 0x7f358806ff00 2024-04-26T19:51:36.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.974+0000 7f358c864640 1 -- 172.21.15.79:0/2338278764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 msgr2=0x7f3588148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:36.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.974+0000 7f358c864640 1 --2- 172.21.15.79:0/2338278764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 0x7f3588148fd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f357c0097a0 tx=0x7f357c02a250 comp rx=0 tx=0).stop 2024-04-26T19:51:36.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.974+0000 7f358c864640 1 -- 172.21.15.79:0/2338278764 shutdown_connections 2024-04-26T19:51:36.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.974+0000 7f358c864640 1 --2- 172.21.15.79:0/2338278764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 0x7f3588148fd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:36.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.974+0000 7f358c864640 1 -- 172.21.15.79:0/2338278764 >> 172.21.15.79:0/2338278764 conn(0x7f3588071c30 msgr2=0x7f358806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:36.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.974+0000 7f358c864640 1 -- 172.21.15.79:0/2338278764 shutdown_connections 2024-04-26T19:51:36.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.974+0000 7f358c864640 1 -- 172.21.15.79:0/2338278764 wait complete. 2024-04-26T19:51:36.978 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.975+0000 7f358c864640 1 Processor -- start 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.975+0000 7f358c864640 1 -- start start 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358c864640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f358806fb20 0x7f3588143e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358c864640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 0x7f358815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358c864640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35880d2cd0 0x7f3588141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358c864640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f35880d4c10 con 0x7f358806fb20 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358c864640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35880d4930 con 0x7f358806ff00 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358c864640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f35880d3920 con 0x7f35880d2cd0 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f358806fb20 0x7f3588143e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f358806fb20 0x7f3588143e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:35190/0 (socket says 172.21.15.79:35190) 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358659c640 1 -- 172.21.15.79:0/3297207731 learned_addr learned my addr 172.21.15.79:0/3297207731 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f3585d9b640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 0x7f358815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:36.979 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f3586d9d640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35880d2cd0 0x7f3588141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358659c640 1 -- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35880d2cd0 msgr2=0x7f3588141ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358659c640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35880d2cd0 0x7f3588141ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358659c640 1 -- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 msgr2=0x7f358815fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358659c640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 0x7f358815fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358659c640 1 -- 172.21.15.79:0/3297207731 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35881604c0 con 0x7f358806fb20 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f3585d9b640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 0x7f358815fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f3586d9d640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35880d2cd0 0x7f3588141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f358659c640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f358806fb20 0x7f3588143e60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f357800c490 tx=0x7f357800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f3576ffd640 1 -- 172.21.15.79:0/3297207731 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35780052d0 con 0x7f358806fb20 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f3576ffd640 1 -- 172.21.15.79:0/3297207731 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3578005470 con 0x7f358806fb20 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.976+0000 7f3576ffd640 1 -- 172.21.15.79:0/3297207731 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3578012850 con 0x7f358806fb20 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.977+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35881607a0 con 0x7f358806fb20 2024-04-26T19:51:36.980 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.977+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3588160d60 con 0x7f358806fb20 2024-04-26T19:51:36.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.978+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35880506c0 con 0x7f358806fb20 2024-04-26T19:51:36.981 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.978+0000 7f3576ffd640 1 -- 172.21.15.79:0/3297207731 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35780129f0 con 0x7f358806fb20 2024-04-26T19:51:36.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.979+0000 7f3576ffd640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f356806c920 0x7f356806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:36.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.979+0000 7f3576ffd640 1 -- 172.21.15.79:0/3297207731 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7f357808b6b0 con 0x7f358806fb20 2024-04-26T19:51:36.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.979+0000 7f3585d9b640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f356806c920 0x7f356806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:36.982 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.980+0000 7f3585d9b640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f356806c920 0x7f356806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f357c034000 tx=0x7f357c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:36.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:36.983+0000 7f3576ffd640 1 -- 172.21.15.79:0/3297207731 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3578059c50 con 0x7f358806fb20 2024-04-26T19:51:37.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.169+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f35881482e0 con 0x7f358806fb20 2024-04-26T19:51:37.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.795+0000 7f3576ffd640 1 -- 172.21.15.79:0/3297207731 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7f3578059550 con 0x7f358806fb20 2024-04-26T19:51:37.796 INFO:teuthology.orchestra.run.smithi079.stderr: removed all blocklist entries 2024-04-26T19:51:37.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f356806c920 msgr2=0x7f356806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:37.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f356806c920 0x7f356806ede0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f357c034000 tx=0x7f357c02e000 comp rx=0 tx=0).stop 2024-04-26T19:51:37.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f358806fb20 msgr2=0x7f3588143e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:37.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f358806fb20 0x7f3588143e60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f357800c490 tx=0x7f357800c950 comp rx=0 tx=0).stop 2024-04-26T19:51:37.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 shutdown_connections 2024-04-26T19:51:37.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f356806c920 0x7f356806ede0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35880d2cd0 0x7f3588141ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f358806ff00 0x7f358815fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 --2- 172.21.15.79:0/3297207731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f358806fb20 0x7f3588143e60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 >> 172.21.15.79:0/3297207731 conn(0x7f3588071c30 msgr2=0x7f358806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:37.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 shutdown_connections 2024-04-26T19:51:37.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.798+0000 7f358c864640 1 -- 172.21.15.79:0/3297207731 wait complete. 2024-04-26T19:51:37.811 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-26T19:51:37.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.908+0000 7f313ac15640 1 Processor -- start 2024-04-26T19:51:37.909 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.908+0000 7f313ac15640 1 -- start start 2024-04-26T19:51:37.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.909+0000 7f313ac15640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f31340cd9c0 0x7f31340b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.909+0000 7f313ac15640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31340ce1a0 con 0x7f31340735a0 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.909+0000 7f313ac15640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31340cdec0 con 0x7f3134156670 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.909+0000 7f313ac15640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3134073110 con 0x7f31340cd9c0 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.909+0000 7f31391b2640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f31340735a0 0x7f31340cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57168/0 (socket says 172.21.15.79:57168) 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.909+0000 7f31391b2640 1 -- 172.21.15.79:0/3637514057 learned_addr learned my addr 172.21.15.79:0/3637514057 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.909+0000 7f3133fff640 1 --2- 172.21.15.79:0/3637514057 >> v2:172.21.15.83:3300/0 conn(0x7f31340cd9c0 0x7f31340b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.909+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1415720550 0 0) 0x7f31340ce1a0 con 0x7f31340735a0 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f311c003520 con 0x7f31340735a0 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4163559105 0 0) 0x7f311c003520 con 0x7f31340735a0 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 >> v2:172.21.15.83:3300/0 conn(0x7f31340cd9c0 msgr2=0x7f31340b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:37.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 --2- 172.21.15.79:0/3637514057 >> v2:172.21.15.83:3300/0 conn(0x7f31340cd9c0 0x7f31340b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 >> v1:172.21.15.83:6789/0 conn(0x7f3134156670 legacy=0x7f313415d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:37.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3134072ac0 con 0x7f31340735a0 2024-04-26T19:51:37.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f312c0019f0 con 0x7f31340735a0 2024-04-26T19:51:37.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f3133fff640 1 --2- 172.21.15.79:0/3637514057 >> v2:172.21.15.83:3300/0 conn(0x7f31340cd9c0 0x7f31340b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:37.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 >> v1:172.21.15.79:6789/0 conn(0x7f31340735a0 legacy=0x7f31340cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:37.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f311c0039b0 0x7f311c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c004800 0x7f311c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f311c0051c0 0x7f311c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3134073110 con 0x7f311c0039b0 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f311c00c060 con 0x7f311c004800 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f311c00c260 con 0x7f311c0051c0 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31389b1640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f311c0039b0 0x7f311c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f3133fff640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c004800 0x7f311c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31391b2640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f311c0051c0 0x7f311c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31389b1640 1 -- 172.21.15.79:0/3637514057 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f311c0051c0 msgr2=0x7f311c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31389b1640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f311c0051c0 0x7f311c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31389b1640 1 -- 172.21.15.79:0/3637514057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c004800 msgr2=0x7f311c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:37.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31389b1640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c004800 0x7f311c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31389b1640 1 -- 172.21.15.79:0/3637514057 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311c003520 con 0x7f311c0039b0 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f31391b2640 1 -- 172.21.15.79:0/3637514057 reap_dead start 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.910+0000 7f3133fff640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c004800 0x7f311c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.911+0000 7f31389b1640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f311c0039b0 0x7f311c003dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3124001750 tx=0x7f311c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3e59f99c14c62b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.911+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124001370 con 0x7f311c0039b0 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.911+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31240052d0 con 0x7f311c0039b0 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.911+0000 7f31337fe640 1 -- 172.21.15.79:0/3637514057 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312402abe0 con 0x7f311c0039b0 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.911+0000 7f313ac15640 1 -- 172.21.15.79:0/3637514057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f311c0039b0 msgr2=0x7f311c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.911+0000 7f313ac15640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f311c0039b0 0x7f311c003dd0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3124001750 tx=0x7f311c005830 comp rx=0 tx=0).stop 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.911+0000 7f313ac15640 1 -- 172.21.15.79:0/3637514057 shutdown_connections 2024-04-26T19:51:37.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.911+0000 7f313ac15640 1 --2- 172.21.15.79:0/3637514057 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f311c0039b0 0x7f311c003dd0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.911+0000 7f313ac15640 1 -- 172.21.15.79:0/3637514057 >> 172.21.15.79:0/3637514057 conn(0x7f31340b39d0 msgr2=0x7f313406feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.912+0000 7f313ac15640 1 -- 172.21.15.79:0/3637514057 shutdown_connections 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.912+0000 7f313ac15640 1 -- 172.21.15.79:0/3637514057 wait complete. 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.913+0000 7f313ac15640 1 Processor -- start 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.913+0000 7f313ac15640 1 -- start start 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.913+0000 7f313ac15640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0039b0 0x7f3134071a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.913+0000 7f313ac15640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f313414dbe0 0x7f313414e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f313ac15640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f313406f810 0x7f313406fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f313ac15640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3134156bd0 con 0x7f313406f810 2024-04-26T19:51:37.925 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f313ac15640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3134156a50 con 0x7f311c0039b0 2024-04-26T19:51:37.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f313ac15640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f313415d130 con 0x7f313414dbe0 2024-04-26T19:51:37.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31391b2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f313406f810 0x7f313406fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:37.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31391b2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f313406f810 0x7f313406fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53786/0 (socket says 172.21.15.79:53786) 2024-04-26T19:51:37.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31391b2640 1 -- 172.21.15.79:0/279790006 learned_addr learned my addr 172.21.15.79:0/279790006 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:37.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f3133fff640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f313414dbe0 0x7f313414e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:37.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31389b1640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0039b0 0x7f3134071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:37.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31391b2640 1 -- 172.21.15.79:0/279790006 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f313414dbe0 msgr2=0x7f313414e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:37.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31391b2640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f313414dbe0 0x7f313414e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31391b2640 1 -- 172.21.15.79:0/279790006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0039b0 msgr2=0x7f3134071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31391b2640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0039b0 0x7f3134071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31391b2640 1 -- 172.21.15.79:0/279790006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313406dea0 con 0x7f313406f810 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31391b2640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f313406f810 0x7f313406fca0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f31340b07c0 tx=0x7f312c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31317fa640 1 -- 172.21.15.79:0/279790006 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c00eb90 con 0x7f313406f810 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31317fa640 1 -- 172.21.15.79:0/279790006 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f312c018070 con 0x7f313406f810 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313406e180 con 0x7f313406f810 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f31317fa640 1 -- 172.21.15.79:0/279790006 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c013420 con 0x7f313406f810 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.914+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313406e740 con 0x7f313406f810 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.915+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31340506c0 con 0x7f313406f810 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.916+0000 7f31317fa640 1 -- 172.21.15.79:0/279790006 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f312c0135c0 con 0x7f313406f810 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.916+0000 7f31317fa640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f312006c8f0 0x7f312006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:37.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.916+0000 7f31317fa640 1 -- 172.21.15.79:0/279790006 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f312c08bab0 con 0x7f313406f810 2024-04-26T19:51:37.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.916+0000 7f31389b1640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f312006c8f0 0x7f312006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:37.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.917+0000 7f31389b1640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f312006c8f0 0x7f312006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3124032000 tx=0x7f312402e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:37.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:37.921+0000 7f31317fa640 1 -- 172.21.15.79:0/279790006 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f312c05a0a0 con 0x7f313406f810 2024-04-26T19:51:38.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.170+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f31340d1590 con 0x7f313406f810 2024-04-26T19:51:38.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.170+0000 7f31317fa640 1 -- 172.21.15.79:0/279790006 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f312c0599a0 con 0x7f313406f810 2024-04-26T19:51:38.171 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:38.171 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:51:38.171 INFO:teuthology.orchestra.run.smithi079.stdout: "auth_dump": [ 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.a", 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBMBSxmKKzNLRAATTamPLUKZz4plkGBTiyOjw==", 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.b", 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBMBSxmOMv9OBAAMfVcb1lc/xGbELkGngAmzw==", 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.172 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.c", 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBMBSxm5rCiNBAAxsM3A75gmGA9hb2dlnJY+Q==", 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.173 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mds.d", 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBNBSxm+y5EBBAAiYAmnzhCCNr2PJ7N7MbH+A==", 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.0", 2024-04-26T19:51:38.174 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBQBSxmUiloIBAAxns7tzts02YP2BVjmjJDyg==", 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.1", 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBTBSxm0GCNHhAAhYZJUvZc2fhkt4UNYgb/7Q==", 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.175 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.2", 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBWBSxmCg42HBAAo+vEqWMyOiC632Saxz/M8Q==", 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.3", 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBZBSxmPx17GhAAYYowgL/L38fF57nmjGtAAw==", 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.176 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.4", 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBfBSxmN3i/IhAA4ctD7r7jlwNMcTKtjc9RWg==", 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.177 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.5", 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBiBSxm5tCnIBAA4uOkO81p3oPAAHhc1iAz8Q==", 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.6", 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBlBSxmfZohHhAAxQw7mbsZ2WNlFq7Dt9MODQ==", 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2024-04-26T19:51:38.178 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "osd.7", 2024-04-26T19:51:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBoBSxmkSVlHBAAfzDuWJJsYr8I4g5we5gQKQ==", 2024-04-26T19:51:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow profile osd", 2024-04-26T19:51:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile osd", 2024-04-26T19:51:38.179 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.0", 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBNBSxmVxwrCBAA6mt2hQ7ZI0Y9b09Qqm6RfQ==", 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.180 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.1", 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBNBSxmeROwEBAAc6r+x9PwvArMxK3fQGtPiw==", 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.2", 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBNBSxm1rmfDRAAL9yuQwmwQGr1wKj6nBaXNQ==", 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2024-04-26T19:51:38.181 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.3", 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBNBSxmi74TFhAA2qTq35nlb+CmAEGuhpIk+Q==", 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow", 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow r", 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow rw", 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow rwx" 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.182 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.admin", 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBMBSxmqCBPExAAr/Kv4HXyTyBBu+m41AQrOg==", 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow *", 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "mgr": "allow *", 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow *", 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-mds", 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBuBSxmYTX6DBAA5VQlWJhaYmfVn5VsDOJvfw==", 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-mds" 2024-04-26T19:51:38.183 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-mgr", 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBuBSxmPz36DBAAtm0voQLxdJiDyw7MNBSSKg==", 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-osd", 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBuBSxmE0T6DBAAaKWFa8/oBtDX2eRLOm6Vnw==", 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-osd" 2024-04-26T19:51:38.184 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-rbd", 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBuBSxmAUz6DBAA06acZrFmU5WUjWmorh/lGQ==", 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBuBSxm/1L6DBAAjapIt3IHlU0+ELXQtdWC7Q==", 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.185 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "client.bootstrap-rgw", 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBuBSxm5Fn6DBAA229Zkhpqo1MI3vvykYH/7A==", 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mgr.x", 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBMBSxmzY+rKhAAqr9zc68jOGYvhstf+AvDXQ==", 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow *", 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile mgr", 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.186 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: }, 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "entity": "mgr.y", 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "key": "AQBMBSxm2bR2JxAAONpqvh+RpLjBgDu2rjiv+Q==", 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "caps": { 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "mds": "allow *", 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "mon": "allow profile mgr", 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: "osd": "allow *" 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout: ] 2024-04-26T19:51:38.187 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:51:38.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f312006c8f0 msgr2=0x7f312006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f312006c8f0 0x7f312006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3124032000 tx=0x7f312402e000 comp rx=0 tx=0).stop 2024-04-26T19:51:38.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f313406f810 msgr2=0x7f313406fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f313406f810 0x7f313406fca0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f31340b07c0 tx=0x7f312c00e530 comp rx=0 tx=0).stop 2024-04-26T19:51:38.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 shutdown_connections 2024-04-26T19:51:38.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f312006c8f0 0x7f312006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f313406f810 0x7f313406fca0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f313414dbe0 0x7f313414e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 --2- 172.21.15.79:0/279790006 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f311c0039b0 0x7f3134071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 >> 172.21.15.79:0/279790006 conn(0x7f31340b39d0 msgr2=0x7f313415b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:38.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 shutdown_connections 2024-04-26T19:51:38.190 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.173+0000 7f313ac15640 1 -- 172.21.15.79:0/279790006 wait complete. 2024-04-26T19:51:38.190 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-26T19:51:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.288+0000 7f10b4693640 1 Processor -- start 2024-04-26T19:51:38.289 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.288+0000 7f10b4693640 1 -- start start 2024-04-26T19:51:38.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b4693640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f10ac06ff00 0x7f10ac148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b4693640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10ac0b1290 con 0x7f10ac0d2cd0 2024-04-26T19:51:38.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b4693640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10ac0d4090 con 0x7f10ac06fb20 2024-04-26T19:51:38.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b4693640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f10ac0d3140 con 0x7f10ac06ff00 2024-04-26T19:51:38.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b1c2e640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f10ac06ff00 0x7f10ac148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b1c2e640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f10ac06ff00 0x7f10ac148fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53802/0 (socket says 172.21.15.79:53802) 2024-04-26T19:51:38.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b1c2e640 1 -- 172.21.15.79:0/3641939935 learned_addr learned my addr 172.21.15.79:0/3641939935 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:38.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b2c30640 1 --1- 172.21.15.79:0/3641939935 >> v1:172.21.15.79:6789/0 conn(0x7f10ac0d2cd0 0x7f10ac0aebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57174/0 (socket says 172.21.15.79:57174) 2024-04-26T19:51:38.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b1c2e640 1 -- 172.21.15.79:0/3641939935 >> v1:172.21.15.83:6789/0 conn(0x7f10ac06fb20 legacy=0x7f10ac1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:38.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b1c2e640 1 -- 172.21.15.79:0/3641939935 >> v1:172.21.15.79:6789/0 conn(0x7f10ac0d2cd0 legacy=0x7f10ac0aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.289+0000 7f10b142d640 1 -- 172.21.15.79:0/3641939935 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2915450161 0 0) 0x7f10ac0b1290 con 0x7f10ac0d2cd0 2024-04-26T19:51:38.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.290+0000 7f10b1c2e640 1 -- 172.21.15.79:0/3641939935 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac0d4370 con 0x7f10ac06ff00 2024-04-26T19:51:38.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.290+0000 7f10b1c2e640 1 --2- 172.21.15.79:0/3641939935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06ff00 0x7f10ac148fd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f109c009870 tx=0x7f109c02a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6869abb842f728 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:38.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.290+0000 7f10b142d640 1 -- 172.21.15.79:0/3641939935 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c037070 con 0x7f10ac06ff00 2024-04-26T19:51:38.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.290+0000 7f10b142d640 1 -- 172.21.15.79:0/3641939935 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109c032640 con 0x7f10ac06ff00 2024-04-26T19:51:38.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.290+0000 7f10b142d640 1 -- 172.21.15.79:0/3641939935 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c032a10 con 0x7f10ac06ff00 2024-04-26T19:51:38.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.291+0000 7f10b4693640 1 -- 172.21.15.79:0/3641939935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06ff00 msgr2=0x7f10ac148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.291+0000 7f10b4693640 1 --2- 172.21.15.79:0/3641939935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06ff00 0x7f10ac148fd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f109c009870 tx=0x7f109c02a330 comp rx=0 tx=0).stop 2024-04-26T19:51:38.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.291+0000 7f10b4693640 1 -- 172.21.15.79:0/3641939935 shutdown_connections 2024-04-26T19:51:38.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.291+0000 7f10b4693640 1 --2- 172.21.15.79:0/3641939935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06ff00 0x7f10ac148fd0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.291+0000 7f10b4693640 1 -- 172.21.15.79:0/3641939935 >> 172.21.15.79:0/3641939935 conn(0x7f10ac071c30 msgr2=0x7f10ac06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:38.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.291+0000 7f10b4693640 1 -- 172.21.15.79:0/3641939935 shutdown_connections 2024-04-26T19:51:38.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.291+0000 7f10b4693640 1 -- 172.21.15.79:0/3641939935 wait complete. 2024-04-26T19:51:38.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.292+0000 7f10b4693640 1 Processor -- start 2024-04-26T19:51:38.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.292+0000 7f10b4693640 1 -- start start 2024-04-26T19:51:38.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b4693640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06fb20 0x7f10ac141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b4693640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f10ac06ff00 0x7f10ac0b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b4693640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10ac0d2cd0 0x7f10ac147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b4693640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac0d4c10 con 0x7f10ac06fb20 2024-04-26T19:51:38.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b4693640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f10ac0d4930 con 0x7f10ac0d2cd0 2024-04-26T19:51:38.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b4693640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f10ac0d3920 con 0x7f10ac06ff00 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b242f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06fb20 0x7f10ac141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b242f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06fb20 0x7f10ac141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53804/0 (socket says 172.21.15.79:53804) 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b242f640 1 -- 172.21.15.79:0/3675195174 learned_addr learned my addr 172.21.15.79:0/3675195174 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b1c2e640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f10ac06ff00 0x7f10ac0b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b2c30640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10ac0d2cd0 0x7f10ac147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b242f640 1 -- 172.21.15.79:0/3675195174 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f10ac06ff00 msgr2=0x7f10ac0b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b242f640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f10ac06ff00 0x7f10ac0b2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b242f640 1 -- 172.21.15.79:0/3675195174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10ac0d2cd0 msgr2=0x7f10ac147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b242f640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10ac0d2cd0 0x7f10ac147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b242f640 1 -- 172.21.15.79:0/3675195174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ac142530 con 0x7f10ac06fb20 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b2c30640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10ac0d2cd0 0x7f10ac147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b242f640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06fb20 0x7f10ac141ae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c7a0 tx=0x7f10a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10a2ffd640 1 -- 172.21.15.79:0/3675195174 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a8017070 con 0x7f10ac06fb20 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10a2ffd640 1 -- 172.21.15.79:0/3675195174 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a8003da0 con 0x7f10ac06fb20 2024-04-26T19:51:38.296 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ac142020 con 0x7f10ac06fb20 2024-04-26T19:51:38.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.293+0000 7f10a2ffd640 1 -- 172.21.15.79:0/3675195174 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a80052b0 con 0x7f10ac06fb20 2024-04-26T19:51:38.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.294+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ac160010 con 0x7f10ac06fb20 2024-04-26T19:51:38.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.294+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ac0506c0 con 0x7f10ac06fb20 2024-04-26T19:51:38.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.295+0000 7f10a2ffd640 1 -- 172.21.15.79:0/3675195174 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10a801a070 con 0x7f10ac06fb20 2024-04-26T19:51:38.297 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.295+0000 7f10a2ffd640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f109006c8f0 0x7f109006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.295+0000 7f10b1c2e640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f109006c8f0 0x7f109006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.295+0000 7f10a2ffd640 1 -- 172.21.15.79:0/3675195174 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f10a808b0b0 con 0x7f10ac06fb20 2024-04-26T19:51:38.298 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.296+0000 7f10b1c2e640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f109006c8f0 0x7f109006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f109c034000 tx=0x7f109c02e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:38.300 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.300+0000 7f10a2ffd640 1 -- 172.21.15.79:0/3675195174 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f10a80596a0 con 0x7f10ac06fb20 2024-04-26T19:51:38.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.540+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f10ac0d3700 con 0x7f10ac06fb20 2024-04-26T19:51:38.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.542+0000 7f10a2ffd640 1 -- 172.21.15.79:0/3675195174 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f10a8058fa0 con 0x7f10ac06fb20 2024-04-26T19:51:38.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f109006c8f0 msgr2=0x7f109006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f109006c8f0 0x7f109006edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f109c034000 tx=0x7f109c02e000 comp rx=0 tx=0).stop 2024-04-26T19:51:38.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06fb20 msgr2=0x7f10ac141ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06fb20 0x7f10ac141ae0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f10a800c7a0 tx=0x7f10a800cc60 comp rx=0 tx=0).stop 2024-04-26T19:51:38.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 shutdown_connections 2024-04-26T19:51:38.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f109006c8f0 0x7f109006edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f10ac0d2cd0 0x7f10ac147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f10ac06ff00 0x7f10ac0b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 --2- 172.21.15.79:0/3675195174 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f10ac06fb20 0x7f10ac141ae0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.544+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 >> 172.21.15.79:0/3675195174 conn(0x7f10ac071c30 msgr2=0x7f10ac0aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:38.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.545+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 shutdown_connections 2024-04-26T19:51:38.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.545+0000 7f10b4693640 1 -- 172.21.15.79:0/3675195174 wait complete. 2024-04-26T19:51:38.556 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-26T19:51:38.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.652+0000 7f9a1bcea640 1 Processor -- start 2024-04-26T19:51:38.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.652+0000 7f9a1bcea640 1 -- start start 2024-04-26T19:51:38.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.653+0000 7f9a1bcea640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9a1406fb20 0x7f9a140d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.653+0000 7f9a1bcea640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a1406ff00 0x7f9a140d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.653+0000 7f9a1bcea640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9a141488c0 0x7f9a14148d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.653+0000 7f9a1bcea640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f9a140d3e60 con 0x7f9a141488c0 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.653+0000 7f9a1bcea640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9a140b1290 con 0x7f9a1406fb20 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.653+0000 7f9a1bcea640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9a140d4090 con 0x7f9a1406ff00 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.653+0000 7f9a1a287640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9a141488c0 0x7f9a14148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.653+0000 7f9a1a287640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f9a141488c0 0x7f9a14148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53814/0 (socket says 172.21.15.79:53814) 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.653+0000 7f9a1a287640 1 -- 172.21.15.79:0/3624129770 learned_addr learned my addr 172.21.15.79:0/3624129770 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a19285640 1 --2- 172.21.15.79:0/3624129770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a1406ff00 0x7f9a140d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a19a86640 1 --2- 172.21.15.79:0/3624129770 >> v2:172.21.15.83:3300/0 conn(0x7f9a1406fb20 0x7f9a140d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a1a287640 1 -- 172.21.15.79:0/3624129770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a1406ff00 msgr2=0x7f9a140d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a1a287640 1 --2- 172.21.15.79:0/3624129770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a1406ff00 0x7f9a140d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a1a287640 1 -- 172.21.15.79:0/3624129770 >> v2:172.21.15.83:3300/0 conn(0x7f9a1406fb20 msgr2=0x7f9a140d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:38.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a1a287640 1 --2- 172.21.15.79:0/3624129770 >> v2:172.21.15.83:3300/0 conn(0x7f9a1406fb20 0x7f9a140d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a1a287640 1 -- 172.21.15.79:0/3624129770 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a140d4370 con 0x7f9a141488c0 2024-04-26T19:51:38.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a19a86640 1 --2- 172.21.15.79:0/3624129770 >> v2:172.21.15.83:3300/0 conn(0x7f9a1406fb20 0x7f9a140d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:38.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a19285640 1 --2- 172.21.15.79:0/3624129770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a1406ff00 0x7f9a140d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:38.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.654+0000 7f9a1a287640 1 --2- 172.21.15.79:0/3624129770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a141488c0 0x7f9a14148d30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9a140706e0 tx=0x7f9a0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc8d2d13ff24da server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:38.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a18a84640 1 -- 172.21.15.79:0/3624129770 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c037070 con 0x7f9a141488c0 2024-04-26T19:51:38.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a18a84640 1 -- 172.21.15.79:0/3624129770 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a0c032440 con 0x7f9a141488c0 2024-04-26T19:51:38.657 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a18a84640 1 -- 172.21.15.79:0/3624129770 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c032810 con 0x7f9a141488c0 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a1bcea640 1 -- 172.21.15.79:0/3624129770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a141488c0 msgr2=0x7f9a14148d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/3624129770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a141488c0 0x7f9a14148d30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9a140706e0 tx=0x7f9a0c02a270 comp rx=0 tx=0).stop 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a1bcea640 1 -- 172.21.15.79:0/3624129770 shutdown_connections 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/3624129770 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a141488c0 0x7f9a14148d30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/3624129770 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a1406ff00 0x7f9a140d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/3624129770 >> v2:172.21.15.83:3300/0 conn(0x7f9a1406fb20 0x7f9a140d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a1bcea640 1 -- 172.21.15.79:0/3624129770 >> 172.21.15.79:0/3624129770 conn(0x7f9a14071c30 msgr2=0x7f9a1406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a1bcea640 1 -- 172.21.15.79:0/3624129770 shutdown_connections 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.655+0000 7f9a1bcea640 1 -- 172.21.15.79:0/3624129770 wait complete. 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.656+0000 7f9a1bcea640 1 Processor -- start 2024-04-26T19:51:38.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.656+0000 7f9a1bcea640 1 -- start start 2024-04-26T19:51:38.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a1bcea640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a1406fb20 0x7f9a141407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a1bcea640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1406ff00 0x7f9a140738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a1bcea640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a141488c0 0x7f9a140b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.661 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a1bcea640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9a140d4c10 con 0x7f9a1406fb20 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a1bcea640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9a140d4930 con 0x7f9a1406ff00 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a1bcea640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9a14149270 con 0x7f9a141488c0 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19a86640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a1406fb20 0x7f9a141407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19a86640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a1406fb20 0x7f9a141407e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53830/0 (socket says 172.21.15.79:53830) 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19a86640 1 -- 172.21.15.79:0/4206892893 learned_addr learned my addr 172.21.15.79:0/4206892893 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19285640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1406ff00 0x7f9a140738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a1a287640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a141488c0 0x7f9a140b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19a86640 1 -- 172.21.15.79:0/4206892893 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a141488c0 msgr2=0x7f9a140b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19a86640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a141488c0 0x7f9a140b3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19a86640 1 -- 172.21.15.79:0/4206892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1406ff00 msgr2=0x7f9a140738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19a86640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1406ff00 0x7f9a140738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19a86640 1 -- 172.21.15.79:0/4206892893 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a14073e30 con 0x7f9a1406fb20 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.657+0000 7f9a19285640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1406ff00 0x7f9a140738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:38.662 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.658+0000 7f9a19a86640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a1406fb20 0x7f9a141407e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007e10 tx=0x7f9a0400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.658+0000 7f9a127fc640 1 -- 172.21.15.79:0/4206892893 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0401a070 con 0x7f9a1406fb20 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.658+0000 7f9a127fc640 1 -- 172.21.15.79:0/4206892893 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a04003b90 con 0x7f9a1406fb20 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.658+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a14140de0 con 0x7f9a1406fb20 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.658+0000 7f9a127fc640 1 -- 172.21.15.79:0/4206892893 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04004030 con 0x7f9a1406fb20 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.658+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a141413a0 con 0x7f9a1406fb20 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.659+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a140506c0 con 0x7f9a1406fb20 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.659+0000 7f9a127fc640 1 -- 172.21.15.79:0/4206892893 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9a0401d070 con 0x7f9a1406fb20 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.659+0000 7f9a127fc640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9a0006c8f0 0x7f9a0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.660+0000 7f9a127fc640 1 -- 172.21.15.79:0/4206892893 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f9a0408b9d0 con 0x7f9a1406fb20 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.660+0000 7f9a19285640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9a0006c8f0 0x7f9a0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:38.663 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.660+0000 7f9a19285640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9a0006c8f0 0x7f9a0006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a1415ef00 tx=0x7f9a08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:38.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.664+0000 7f9a127fc640 1 -- 172.21.15.79:0/4206892893 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9a04010070 con 0x7f9a1406fb20 2024-04-26T19:51:38.910 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.910+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f9a140d3700 con 0x7f9a1406fb20 2024-04-26T19:51:38.912 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.912+0000 7f9a127fc640 1 -- 172.21.15.79:0/4206892893 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f9a04059a00 con 0x7f9a1406fb20 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9a0006c8f0 msgr2=0x7f9a0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9a0006c8f0 0x7f9a0006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f9a1415ef00 tx=0x7f9a08008040 comp rx=0 tx=0).stop 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a1406fb20 msgr2=0x7f9a141407e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a1406fb20 0x7f9a141407e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9a04007e10 tx=0x7f9a0400d4a0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 shutdown_connections 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f9a0006c8f0 0x7f9a0006edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9a141488c0 0x7f9a140b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9a1406ff00 0x7f9a140738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 --2- 172.21.15.79:0/4206892893 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9a1406fb20 0x7f9a141407e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 >> 172.21.15.79:0/4206892893 conn(0x7f9a14071c30 msgr2=0x7f9a140d2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:38.915 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 shutdown_connections 2024-04-26T19:51:38.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:38.914+0000 7f9a1bcea640 1 -- 172.21.15.79:0/4206892893 wait complete. 2024-04-26T19:51:38.925 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-26T19:51:39.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.022+0000 7f4bb59f0640 1 Processor -- start 2024-04-26T19:51:39.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.022+0000 7f4bb59f0640 1 -- start start 2024-04-26T19:51:39.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bb59f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb013fee0 0x7f4bb01432c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bb59f0640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4bb01402c0 0x7f4bb00b3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bb59f0640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb0143800 con 0x7f4bb00b4080 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bb59f0640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb00b44f0 con 0x7f4bb01402c0 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bb59f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb00aa880 con 0x7f4bb013fee0 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bae7fc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4bb01402c0 0x7f4bb00b3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4baf7fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4bb00b4080 0x7f4bb0160e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57184/0 (socket says 172.21.15.79:57184) 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bae7fc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4bb01402c0 0x7f4bb00b3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53846/0 (socket says 172.21.15.79:53846) 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4baf7fe640 1 -- 172.21.15.79:0/1806020183 learned_addr learned my addr 172.21.15.79:0/1806020183 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4baeffd640 1 --2- 172.21.15.79:0/1806020183 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb013fee0 0x7f4bb01432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bae7fc640 1 -- 172.21.15.79:0/1806020183 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb013fee0 msgr2=0x7f4bb01432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bae7fc640 1 --2- 172.21.15.79:0/1806020183 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb013fee0 0x7f4bb01432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bae7fc640 1 -- 172.21.15.79:0/1806020183 >> v1:172.21.15.79:6789/0 conn(0x7f4bb00b4080 legacy=0x7f4bb0160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4badffb640 1 -- 172.21.15.79:0/1806020183 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317768203 0 0) 0x7f4bb0143800 con 0x7f4bb00b4080 2024-04-26T19:51:39.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.023+0000 7f4bae7fc640 1 -- 172.21.15.79:0/1806020183 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb00aab60 con 0x7f4bb01402c0 2024-04-26T19:51:39.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.024+0000 7f4bae7fc640 1 --2- 172.21.15.79:0/1806020183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb01402c0 0x7f4bb00b3b40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40097a0 tx=0x7f4ba402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3205a7e65a8e32bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:39.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.024+0000 7f4badffb640 1 -- 172.21.15.79:0/1806020183 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4037070 con 0x7f4bb01402c0 2024-04-26T19:51:39.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.024+0000 7f4badffb640 1 -- 172.21.15.79:0/1806020183 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba4032440 con 0x7f4bb01402c0 2024-04-26T19:51:39.026 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.024+0000 7f4badffb640 1 -- 172.21.15.79:0/1806020183 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba4032810 con 0x7f4bb01402c0 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.025+0000 7f4bb59f0640 1 -- 172.21.15.79:0/1806020183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb01402c0 msgr2=0x7f4bb00b3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.025+0000 7f4bb59f0640 1 --2- 172.21.15.79:0/1806020183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb01402c0 0x7f4bb00b3b40 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40097a0 tx=0x7f4ba402a180 comp rx=0 tx=0).stop 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.025+0000 7f4bb59f0640 1 -- 172.21.15.79:0/1806020183 shutdown_connections 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.025+0000 7f4bb59f0640 1 --2- 172.21.15.79:0/1806020183 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb01402c0 0x7f4bb00b3b40 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.025+0000 7f4bb59f0640 1 --2- 172.21.15.79:0/1806020183 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb013fee0 0x7f4bb01432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.025+0000 7f4bb59f0640 1 -- 172.21.15.79:0/1806020183 >> 172.21.15.79:0/1806020183 conn(0x7f4bb00b2720 msgr2=0x7f4bb0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.025+0000 7f4bb59f0640 1 -- 172.21.15.79:0/1806020183 shutdown_connections 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.025+0000 7f4bb59f0640 1 -- 172.21.15.79:0/1806020183 wait complete. 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.026+0000 7f4bb59f0640 1 Processor -- start 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.026+0000 7f4bb59f0640 1 -- start start 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4bb59f0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb00b4080 0x7f4bb015f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4bb59f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bb013fee0 0x7f4bb013dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4bb59f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb01402c0 0x7f4bb01f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4bb59f0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb00ab400 con 0x7f4bb00b4080 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4bb59f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb00ab120 con 0x7f4bb013fee0 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4bb59f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb00d29c0 con 0x7f4bb01402c0 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb00b4080 0x7f4bb015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baeffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb00b4080 0x7f4bb015f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53862/0 (socket says 172.21.15.79:53862) 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baeffd640 1 -- 172.21.15.79:0/882458755 learned_addr learned my addr 172.21.15.79:0/882458755 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baeffd640 1 -- 172.21.15.79:0/882458755 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb01402c0 msgr2=0x7f4bb01f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baf7fe640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb01402c0 0x7f4bb01f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baeffd640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb01402c0 0x7f4bb01f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baeffd640 1 -- 172.21.15.79:0/882458755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bb013fee0 msgr2=0x7f4bb013dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baeffd640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bb013fee0 0x7f4bb013dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baeffd640 1 -- 172.21.15.79:0/882458755 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb006c6d0 con 0x7f4bb00b4080 2024-04-26T19:51:39.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4baeffd640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb00b4080 0x7f4bb015f790 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0007ef0 tx=0x7f4ba000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:39.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4b97fff640 1 -- 172.21.15.79:0/882458755 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0014620 con 0x7f4bb00b4080 2024-04-26T19:51:39.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4b97fff640 1 -- 172.21.15.79:0/882458755 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba0019070 con 0x7f4bb00b4080 2024-04-26T19:51:39.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb006c9b0 con 0x7f4bb00b4080 2024-04-26T19:51:39.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4b97fff640 1 -- 172.21.15.79:0/882458755 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0014a00 con 0x7f4bb00b4080 2024-04-26T19:51:39.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.027+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb006cf70 con 0x7f4bb00b4080 2024-04-26T19:51:39.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.028+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb00506c0 con 0x7f4bb00b4080 2024-04-26T19:51:39.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.029+0000 7f4b97fff640 1 -- 172.21.15.79:0/882458755 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ba0014ba0 con 0x7f4bb00b4080 2024-04-26T19:51:39.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.029+0000 7f4b97fff640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.029+0000 7f4bae7fc640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.029+0000 7f4b97fff640 1 -- 172.21.15.79:0/882458755 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f4ba008c210 con 0x7f4bb00b4080 2024-04-26T19:51:39.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.030+0000 7f4bae7fc640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4034000 tx=0x7f4ba402c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:39.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.034+0000 7f4b97fff640 1 -- 172.21.15.79:0/882458755 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4ba005a890 con 0x7f4bb00b4080 2024-04-26T19:51:39.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.275+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f4bb0140b10 con 0x7f4bb00b4080 2024-04-26T19:51:39.277 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.277+0000 7f4b97fff640 1 -- 172.21.15.79:0/882458755 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f4ba005a190 con 0x7f4bb00b4080 2024-04-26T19:51:39.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b9c06c8f0 msgr2=0x7f4b9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.279 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4034000 tx=0x7f4ba402c000 comp rx=0 tx=0).stop 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb00b4080 msgr2=0x7f4bb015f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb00b4080 0x7f4bb015f790 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4ba0007ef0 tx=0x7f4ba000e4b0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 shutdown_connections 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4bb01402c0 0x7f4bb01f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4bb013fee0 0x7f4bb013dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 --2- 172.21.15.79:0/882458755 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4bb00b4080 0x7f4bb015f790 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 >> 172.21.15.79:0/882458755 conn(0x7f4bb00b2720 msgr2=0x7f4bb0070670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 shutdown_connections 2024-04-26T19:51:39.280 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.279+0000 7f4bb59f0640 1 -- 172.21.15.79:0/882458755 wait complete. 2024-04-26T19:51:39.291 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-26T19:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.389+0000 7f0212e99640 1 Processor -- start 2024-04-26T19:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.389+0000 7f0212e99640 1 -- start start 2024-04-26T19:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.389+0000 7f0212e99640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f020c160390 0x7f020c0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.389+0000 7f0212e99640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f020c160770 0x7f020c0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.389+0000 7f0212e99640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c1630d0 0x7f020c163560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.389+0000 7f0212e99640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f020c144ca0 con 0x7f020c160770 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.389+0000 7f0212e99640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f020c163e00 con 0x7f020c160390 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.389+0000 7f0212e99640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f020c144f30 con 0x7f020c1630d0 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f020bfff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f020c160770 0x7f020c0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f020bfff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f020c160770 0x7f020c0b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53878/0 (socket says 172.21.15.79:53878) 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f020bfff640 1 -- 172.21.15.79:0/1069921879 learned_addr learned my addr 172.21.15.79:0/1069921879 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f0211436640 1 --2- 172.21.15.79:0/1069921879 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c1630d0 0x7f020c163560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f0210c35640 1 --2- 172.21.15.79:0/1069921879 >> v2:172.21.15.83:3300/0 conn(0x7f020c160390 0x7f020c0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f020bfff640 1 -- 172.21.15.79:0/1069921879 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c1630d0 msgr2=0x7f020c163560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f020bfff640 1 --2- 172.21.15.79:0/1069921879 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c1630d0 0x7f020c163560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f020bfff640 1 -- 172.21.15.79:0/1069921879 >> v2:172.21.15.83:3300/0 conn(0x7f020c160390 msgr2=0x7f020c0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f020bfff640 1 --2- 172.21.15.79:0/1069921879 >> v2:172.21.15.83:3300/0 conn(0x7f020c160390 0x7f020c0b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.390+0000 7f020bfff640 1 -- 172.21.15.79:0/1069921879 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c145210 con 0x7f020c160770 2024-04-26T19:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f020bfff640 1 --2- 172.21.15.79:0/1069921879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 0x7f020c0b4050 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f020c0b5690 tx=0x7f01f402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8953c4f91004b362 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f020b7fe640 1 -- 172.21.15.79:0/1069921879 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4037070 con 0x7f020c160770 2024-04-26T19:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f020b7fe640 1 -- 172.21.15.79:0/1069921879 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f4032440 con 0x7f020c160770 2024-04-26T19:51:39.393 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f020b7fe640 1 -- 172.21.15.79:0/1069921879 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4032810 con 0x7f020c160770 2024-04-26T19:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f0212e99640 1 -- 172.21.15.79:0/1069921879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 msgr2=0x7f020c0b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f0212e99640 1 --2- 172.21.15.79:0/1069921879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 0x7f020c0b4050 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f020c0b5690 tx=0x7f01f402a140 comp rx=0 tx=0).stop 2024-04-26T19:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f0212e99640 1 -- 172.21.15.79:0/1069921879 shutdown_connections 2024-04-26T19:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f0212e99640 1 --2- 172.21.15.79:0/1069921879 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c1630d0 0x7f020c163560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f0212e99640 1 --2- 172.21.15.79:0/1069921879 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 0x7f020c0b4050 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f0212e99640 1 --2- 172.21.15.79:0/1069921879 >> v2:172.21.15.83:3300/0 conn(0x7f020c160390 0x7f020c0b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f0212e99640 1 -- 172.21.15.79:0/1069921879 >> 172.21.15.79:0/1069921879 conn(0x7f020c0b2720 msgr2=0x7f020c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f0212e99640 1 -- 172.21.15.79:0/1069921879 shutdown_connections 2024-04-26T19:51:39.396 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.391+0000 7f0212e99640 1 -- 172.21.15.79:0/1069921879 wait complete. 2024-04-26T19:51:39.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.392+0000 7f0212e99640 1 Processor -- start 2024-04-26T19:51:39.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.392+0000 7f0212e99640 1 -- start start 2024-04-26T19:51:39.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.393+0000 7f0212e99640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c160390 0x7f020c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.393+0000 7f0212e99640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 0x7f020c147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.393+0000 7f0212e99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020c1630d0 0x7f020c06f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.393+0000 7f0212e99640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f020c145a10 con 0x7f020c160770 2024-04-26T19:51:39.397 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.393+0000 7f0212e99640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f020c163aa0 con 0x7f020c1630d0 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.393+0000 7f0212e99640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f020c145b90 con 0x7f020c160390 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.393+0000 7f020bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 0x7f020c147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.393+0000 7f020bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 0x7f020c147720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53894/0 (socket says 172.21.15.79:53894) 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.393+0000 7f020bfff640 1 -- 172.21.15.79:0/3948136644 learned_addr learned my addr 172.21.15.79:0/3948136644 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f0210c35640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c160390 0x7f020c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f0211436640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020c1630d0 0x7f020c06f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f020bfff640 1 -- 172.21.15.79:0/3948136644 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c160390 msgr2=0x7f020c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f020bfff640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c160390 0x7f020c158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f020bfff640 1 -- 172.21.15.79:0/3948136644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020c1630d0 msgr2=0x7f020c06f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f020bfff640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020c1630d0 0x7f020c06f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f020bfff640 1 -- 172.21.15.79:0/3948136644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020c15f210 con 0x7f020c160770 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f020bfff640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 0x7f020c147720 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f01f402a180 tx=0x7f01f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:39.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f02097fa640 1 -- 172.21.15.79:0/3948136644 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4041020 con 0x7f020c160770 2024-04-26T19:51:39.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f02097fa640 1 -- 172.21.15.79:0/3948136644 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f40306c0 con 0x7f020c160770 2024-04-26T19:51:39.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f02097fa640 1 -- 172.21.15.79:0/3948136644 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f4037040 con 0x7f020c160770 2024-04-26T19:51:39.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020c15f4f0 con 0x7f020c160770 2024-04-26T19:51:39.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.394+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f020c15fab0 con 0x7f020c160770 2024-04-26T19:51:39.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.395+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020c0b5920 con 0x7f020c160770 2024-04-26T19:51:39.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.395+0000 7f02097fa640 1 -- 172.21.15.79:0/3948136644 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01f4044020 con 0x7f020c160770 2024-04-26T19:51:39.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.396+0000 7f02097fa640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f01ec06c8f0 0x7f01ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.396+0000 7f02097fa640 1 -- 172.21.15.79:0/3948136644 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f01f40ac190 con 0x7f020c160770 2024-04-26T19:51:39.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.396+0000 7f0210c35640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f01ec06c8f0 0x7f01ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.396+0000 7f0210c35640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f01ec06c8f0 0x7f01ec06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0204004450 tx=0x7f0204003a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:39.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.400+0000 7f02097fa640 1 -- 172.21.15.79:0/3948136644 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f020c0b5920 con 0x7f020c160770 2024-04-26T19:51:39.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.643+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f020c0506c0 con 0x7f020c160770 2024-04-26T19:51:39.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.645+0000 7f02097fa640 1 -- 172.21.15.79:0/3948136644 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f01f404b030 con 0x7f020c160770 2024-04-26T19:51:39.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f01ec06c8f0 msgr2=0x7f01ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f01ec06c8f0 0x7f01ec06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0204004450 tx=0x7f0204003a30 comp rx=0 tx=0).stop 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 msgr2=0x7f020c147720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 0x7f020c147720 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f01f402a180 tx=0x7f01f40046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 shutdown_connections 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f01ec06c8f0 0x7f01ec06edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f020c1630d0 0x7f020c06f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f020c160770 0x7f020c147720 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 --2- 172.21.15.79:0/3948136644 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f020c160390 0x7f020c158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 >> 172.21.15.79:0/3948136644 conn(0x7f020c0b2720 msgr2=0x7f020c140910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 shutdown_connections 2024-04-26T19:51:39.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.648+0000 7f0212e99640 1 -- 172.21.15.79:0/3948136644 wait complete. 2024-04-26T19:51:39.659 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-26T19:51:39.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.755+0000 7fd6718c4640 1 Processor -- start 2024-04-26T19:51:39.756 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.756+0000 7fd6718c4640 1 -- start start 2024-04-26T19:51:39.757 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.756+0000 7fd6718c4640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd66c06ff00 0x7fd66c148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.756+0000 7fd6718c4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd66c0d2cd0 0x7fd66c0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.756+0000 7fd6718c4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd66c0b1290 con 0x7fd66c06fb20 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.756+0000 7fd6718c4640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd66c0d3140 con 0x7fd66c06ff00 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.756+0000 7fd6718c4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd66c0d3e60 con 0x7fd66c0d2cd0 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.757+0000 7fd66a7fc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd66c06ff00 0x7fd66c148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.757+0000 7fd66a7fc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd66c06ff00 0x7fd66c148fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53900/0 (socket says 172.21.15.79:53900) 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.757+0000 7fd66a7fc640 1 -- 172.21.15.79:0/3288489717 learned_addr learned my addr 172.21.15.79:0/3288489717 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.757+0000 7fd66b7fe640 1 --2- 172.21.15.79:0/3288489717 >> v2:172.21.15.83:3300/0 conn(0x7fd66c0d2cd0 0x7fd66c0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.757+0000 7fd66a7fc640 1 -- 172.21.15.79:0/3288489717 >> v2:172.21.15.83:3300/0 conn(0x7fd66c0d2cd0 msgr2=0x7fd66c0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.757+0000 7fd66a7fc640 1 --2- 172.21.15.79:0/3288489717 >> v2:172.21.15.83:3300/0 conn(0x7fd66c0d2cd0 0x7fd66c0d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.757+0000 7fd66a7fc640 1 -- 172.21.15.79:0/3288489717 >> v1:172.21.15.83:6789/0 conn(0x7fd66c06fb20 legacy=0x7fd66c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:39.758 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.757+0000 7fd66a7fc640 1 -- 172.21.15.79:0/3288489717 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c0d4090 con 0x7fd66c06ff00 2024-04-26T19:51:39.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd66a7fc640 1 --2- 172.21.15.79:0/3288489717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06ff00 0x7fd66c148fd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0b0af0 tx=0x7fd66002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f412f90c439c7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:39.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd669ffb640 1 -- 172.21.15.79:0/3288489717 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660037070 con 0x7fd66c06ff00 2024-04-26T19:51:39.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd669ffb640 1 -- 172.21.15.79:0/3288489717 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd660032440 con 0x7fd66c06ff00 2024-04-26T19:51:39.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd669ffb640 1 -- 172.21.15.79:0/3288489717 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd660032810 con 0x7fd66c06ff00 2024-04-26T19:51:39.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd6718c4640 1 -- 172.21.15.79:0/3288489717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06ff00 msgr2=0x7fd66c148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd6718c4640 1 --2- 172.21.15.79:0/3288489717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06ff00 0x7fd66c148fd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0b0af0 tx=0x7fd66002a2c0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.760 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd6718c4640 1 -- 172.21.15.79:0/3288489717 shutdown_connections 2024-04-26T19:51:39.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd6718c4640 1 --2- 172.21.15.79:0/3288489717 >> v2:172.21.15.83:3300/0 conn(0x7fd66c0d2cd0 0x7fd66c0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd6718c4640 1 --2- 172.21.15.79:0/3288489717 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06ff00 0x7fd66c148fd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd6718c4640 1 -- 172.21.15.79:0/3288489717 >> 172.21.15.79:0/3288489717 conn(0x7fd66c071c30 msgr2=0x7fd66c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:39.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd6718c4640 1 -- 172.21.15.79:0/3288489717 shutdown_connections 2024-04-26T19:51:39.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.758+0000 7fd6718c4640 1 -- 172.21.15.79:0/3288489717 wait complete. 2024-04-26T19:51:39.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.759+0000 7fd6718c4640 1 Processor -- start 2024-04-26T19:51:39.761 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd6718c4640 1 -- start start 2024-04-26T19:51:39.765 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd6718c4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06fb20 0x7fd66c1604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd6718c4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd66c06ff00 0x7fd66c0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd6718c4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd66c0d2cd0 0x7fd66c1f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd6718c4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0d4930 con 0x7fd66c06fb20 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd6718c4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd66c0d4650 con 0x7fd66c0d2cd0 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd6718c4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd66c0d4c10 con 0x7fd66c06ff00 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd66affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06fb20 0x7fd66c1604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd66affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06fb20 0x7fd66c1604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53902/0 (socket says 172.21.15.79:53902) 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.760+0000 7fd66affd640 1 -- 172.21.15.79:0/1797830559 learned_addr learned my addr 172.21.15.79:0/1797830559 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd66a7fc640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd66c06ff00 0x7fd66c0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd66b7fe640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd66c0d2cd0 0x7fd66c1f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd66affd640 1 -- 172.21.15.79:0/1797830559 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd66c06ff00 msgr2=0x7fd66c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.766 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd66affd640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd66c06ff00 0x7fd66c0b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd66affd640 1 -- 172.21.15.79:0/1797830559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd66c0d2cd0 msgr2=0x7fd66c1f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd66affd640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd66c0d2cd0 0x7fd66c1f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd66affd640 1 -- 172.21.15.79:0/1797830559 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd66c1f5ba0 con 0x7fd66c06fb20 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd66affd640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06fb20 0x7fd66c1604d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c430 tx=0x7fd65c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd653fff640 1 -- 172.21.15.79:0/1797830559 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c011790 con 0x7fd66c06fb20 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd653fff640 1 -- 172.21.15.79:0/1797830559 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd65c017070 con 0x7fd66c06fb20 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd66c1f5e80 con 0x7fd66c06fb20 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd653fff640 1 -- 172.21.15.79:0/1797830559 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c011bb0 con 0x7fd66c06fb20 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.761+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd66c15ffb0 con 0x7fd66c06fb20 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.762+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd66c1f6050 con 0x7fd66c06fb20 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.762+0000 7fd653fff640 1 -- 172.21.15.79:0/1797830559 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd65c01c460 con 0x7fd66c06fb20 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.763+0000 7fd653fff640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd65806c8f0 0x7fd65806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.763+0000 7fd653fff640 1 -- 172.21.15.79:0/1797830559 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fd65c08b670 con 0x7fd66c06fb20 2024-04-26T19:51:39.767 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.763+0000 7fd66a7fc640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd65806c8f0 0x7fd65806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:39.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.763+0000 7fd66a7fc640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd65806c8f0 0x7fd65806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd660034000 tx=0x7fd66002e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:39.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:39.767+0000 7fd653fff640 1 -- 172.21.15.79:0/1797830559 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd65c0596a0 con 0x7fd66c06fb20 2024-04-26T19:51:40.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.010+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fd66c0506c0 con 0x7fd66c06fb20 2024-04-26T19:51:40.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.015+0000 7fd653fff640 1 -- 172.21.15.79:0/1797830559 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fd65c05d9a0 con 0x7fd66c06fb20 2024-04-26T19:51:40.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd65806c8f0 msgr2=0x7fd65806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd65806c8f0 0x7fd65806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd660034000 tx=0x7fd66002e000 comp rx=0 tx=0).stop 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06fb20 msgr2=0x7fd66c1604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06fb20 0x7fd66c1604d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00c430 tx=0x7fd65c00c8f0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 shutdown_connections 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fd65806c8f0 0x7fd65806edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd66c0d2cd0 0x7fd66c1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd66c06ff00 0x7fd66c0b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 --2- 172.21.15.79:0/1797830559 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd66c06fb20 0x7fd66c1604d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 >> 172.21.15.79:0/1797830559 conn(0x7fd66c071c30 msgr2=0x7fd66c0b3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 shutdown_connections 2024-04-26T19:51:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.017+0000 7fd6718c4640 1 -- 172.21.15.79:0/1797830559 wait complete. 2024-04-26T19:51:40.028 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-26T19:51:40.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.125+0000 7f36026f1640 1 Processor -- start 2024-04-26T19:51:40.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.125+0000 7f36026f1640 1 -- start start 2024-04-26T19:51:40.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f36026f1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f35fc160390 0x7f35fc0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f36026f1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35fc144ca0 con 0x7f35fc160770 2024-04-26T19:51:40.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f36026f1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35fc163e00 con 0x7f35fc1637e0 2024-04-26T19:51:40.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f36026f1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f35fc144f30 con 0x7f35fc160390 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35fb7fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f35fc160770 0x7f35fc1630d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57192/0 (socket says 172.21.15.79:57192) 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35fb7fe640 1 -- 172.21.15.79:0/1468681149 learned_addr learned my addr 172.21.15.79:0/1468681149 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35fbfff640 1 --2- 172.21.15.79:0/1468681149 >> v2:172.21.15.83:3300/0 conn(0x7f35fc160390 0x7f35fc0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631107926 0 0) 0x7f35fc144ca0 con 0x7f35fc160770 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f35e4003520 con 0x7f35fc160770 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 973610486 0 0) 0x7f35e4003520 con 0x7f35fc160770 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 >> v2:172.21.15.83:3300/0 conn(0x7f35fc160390 msgr2=0x7f35fc0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 --2- 172.21.15.79:0/1468681149 >> v2:172.21.15.83:3300/0 conn(0x7f35fc160390 0x7f35fc0b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 >> v1:172.21.15.83:6789/0 conn(0x7f35fc1637e0 legacy=0x7f35fc169460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc145210 con 0x7f35fc160770 2024-04-26T19:51:40.130 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f35f0001670 con 0x7f35fc160770 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 >> v1:172.21.15.79:6789/0 conn(0x7f35fc160770 legacy=0x7f35fc1630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35e40039b0 0x7f35e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35e4004800 0x7f35e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.126+0000 7f35faffd640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e40051c0 0x7f35e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc144f30 con 0x7f35e40051c0 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35e400c160 con 0x7f35e40039b0 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f35e400c360 con 0x7f35e4004800 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f3600c8e640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e40051c0 0x7f35e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f35fbfff640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35e40039b0 0x7f35e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f35fb7fe640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35e4004800 0x7f35e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f3600c8e640 1 -- 172.21.15.79:0/1468681149 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35e4004800 msgr2=0x7f35e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f3600c8e640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35e4004800 0x7f35e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f3600c8e640 1 -- 172.21.15.79:0/1468681149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35e40039b0 msgr2=0x7f35e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:40.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f3600c8e640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35e40039b0 0x7f35e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f3600c8e640 1 -- 172.21.15.79:0/1468681149 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e4003520 con 0x7f35e40051c0 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.127+0000 7f3600c8e640 1 -- 172.21.15.79:0/1468681149 reap_dead start 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f3600c8e640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e40051c0 0x7f35e400b9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f35f4001e50 tx=0x7f35e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675112a259f9fb0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4001a70 con 0x7f35e40051c0 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f4003970 con 0x7f35e40051c0 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f35faffd640 1 -- 172.21.15.79:0/1468681149 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f4003d60 con 0x7f35e40051c0 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f36026f1640 1 -- 172.21.15.79:0/1468681149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e40051c0 msgr2=0x7f35e400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f36026f1640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e40051c0 0x7f35e400b9c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f35f4001e50 tx=0x7f35e4005830 comp rx=0 tx=0).stop 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f36026f1640 1 -- 172.21.15.79:0/1468681149 shutdown_connections 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f36026f1640 1 --2- 172.21.15.79:0/1468681149 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35e40051c0 0x7f35e400b9c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f36026f1640 1 -- 172.21.15.79:0/1468681149 >> 172.21.15.79:0/1468681149 conn(0x7f35fc0b2720 msgr2=0x7f35fc070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f36026f1640 1 -- 172.21.15.79:0/1468681149 shutdown_connections 2024-04-26T19:51:40.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.128+0000 7f36026f1640 1 -- 172.21.15.79:0/1468681149 wait complete. 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.129+0000 7f36026f1640 1 Processor -- start 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f36026f1640 1 -- start start 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f36026f1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35e40051c0 0x7f35fc15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f36026f1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35fc159630 0x7f35fc06f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f36026f1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35fc159a90 0x7f35fc06fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f36026f1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc145b90 con 0x7f35fc159630 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f36026f1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f35fc145a10 con 0x7f35fc159a90 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f36026f1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f35fc1630d0 con 0x7f35e40051c0 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f35fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35fc159630 0x7f35fc06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f35fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35fc159630 0x7f35fc06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53928/0 (socket says 172.21.15.79:53928) 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.130+0000 7f35fb7fe640 1 -- 172.21.15.79:0/2605847062 learned_addr learned my addr 172.21.15.79:0/2605847062 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:40.136 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35fbfff640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35e40051c0 0x7f35fc15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f3600c8e640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35fc159a90 0x7f35fc06fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35fb7fe640 1 -- 172.21.15.79:0/2605847062 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35e40051c0 msgr2=0x7f35fc15a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35fb7fe640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35e40051c0 0x7f35fc15a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35fb7fe640 1 -- 172.21.15.79:0/2605847062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35fc159a90 msgr2=0x7f35fc06fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35fb7fe640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35fc159a90 0x7f35fc06fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35fb7fe640 1 -- 172.21.15.79:0/2605847062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35fc160e20 con 0x7f35fc159630 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f3600c8e640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35fc159a90 0x7f35fc06fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35fb7fe640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35fc159630 0x7f35fc06f930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f35f00072f0 tx=0x7f35f00077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35f8ff9640 1 -- 172.21.15.79:0/2605847062 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f000f960 con 0x7f35fc159630 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35f8ff9640 1 -- 172.21.15.79:0/2605847062 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f0018070 con 0x7f35fc159630 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35fc161100 con 0x7f35fc159630 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f35f8ff9640 1 -- 172.21.15.79:0/2605847062 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f000fd40 con 0x7f35fc159630 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.131+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35fc1616c0 con 0x7f35fc159630 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.132+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35fc0506c0 con 0x7f35fc159630 2024-04-26T19:51:40.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.132+0000 7f35f8ff9640 1 -- 172.21.15.79:0/2605847062 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35f001d460 con 0x7f35fc159630 2024-04-26T19:51:40.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.133+0000 7f35f8ff9640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f35e806c920 0x7f35e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.133+0000 7f35f8ff9640 1 -- 172.21.15.79:0/2605847062 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f35f008ba00 con 0x7f35fc159630 2024-04-26T19:51:40.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.133+0000 7f35fbfff640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f35e806c920 0x7f35e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.133+0000 7f35fbfff640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f35e806c920 0x7f35e806ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009980 tx=0x7f35ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:40.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.137+0000 7f35f8ff9640 1 -- 172.21.15.79:0/2605847062 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f35f0059ff0 con 0x7f35fc159630 2024-04-26T19:51:40.381 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.381+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f35fc0b2500 con 0x7f35fc159630 2024-04-26T19:51:40.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.385+0000 7f35f8ff9640 1 -- 172.21.15.79:0/2605847062 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f35f00598f0 con 0x7f35fc159630 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f35e806c920 msgr2=0x7f35e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f35e806c920 0x7f35e806ede0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f35ec009980 tx=0x7f35ec008040 comp rx=0 tx=0).stop 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35fc159630 msgr2=0x7f35fc06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35fc159630 0x7f35fc06f930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f35f00072f0 tx=0x7f35f00077b0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 shutdown_connections 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f35e806c920 0x7f35e806ede0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f35fc159a90 0x7f35fc06fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f35fc159630 0x7f35fc06f930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 --2- 172.21.15.79:0/2605847062 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f35e40051c0 0x7f35fc15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 >> 172.21.15.79:0/2605847062 conn(0x7f35fc0b2720 msgr2=0x7f35fc0b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 shutdown_connections 2024-04-26T19:51:40.388 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.387+0000 7f36026f1640 1 -- 172.21.15.79:0/2605847062 wait complete. 2024-04-26T19:51:40.398 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:40.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.494+0000 7f3cf9140640 1 Processor -- start 2024-04-26T19:51:40.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.494+0000 7f3cf9140640 1 -- start start 2024-04-26T19:51:40.498 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf9140640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cf4160390 0x7f3cf40b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf9140640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3cf4160770 0x7f3cf40b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf9140640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cf4144ca0 con 0x7f3cf41630d0 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf9140640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf4163e00 con 0x7f3cf4160770 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf9140640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3cf4144f30 con 0x7f3cf4160390 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf259c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3cf4160770 0x7f3cf40b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf259c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3cf4160770 0x7f3cf40b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53930/0 (socket says 172.21.15.79:53930) 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf259c640 1 -- 172.21.15.79:0/2254783306 learned_addr learned my addr 172.21.15.79:0/2254783306 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf2d9d640 1 --2- 172.21.15.79:0/2254783306 >> v2:172.21.15.83:3300/0 conn(0x7f3cf4160390 0x7f3cf40b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf259c640 1 -- 172.21.15.79:0/2254783306 >> v2:172.21.15.83:3300/0 conn(0x7f3cf4160390 msgr2=0x7f3cf40b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf259c640 1 --2- 172.21.15.79:0/2254783306 >> v2:172.21.15.83:3300/0 conn(0x7f3cf4160390 0x7f3cf40b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf259c640 1 -- 172.21.15.79:0/2254783306 >> v1:172.21.15.83:6789/0 conn(0x7f3cf41630d0 legacy=0x7f3cf4163520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf259c640 1 -- 172.21.15.79:0/2254783306 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf4145210 con 0x7f3cf4160770 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.495+0000 7f3cf2d9d640 1 --2- 172.21.15.79:0/2254783306 >> v2:172.21.15.83:3300/0 conn(0x7f3cf4160390 0x7f3cf40b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:40.499 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf259c640 1 --2- 172.21.15.79:0/2254783306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 0x7f3cf40b4050 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40b5690 tx=0x7f3ce802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5662334d38a4dc1c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf1d9b640 1 -- 172.21.15.79:0/2254783306 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8037070 con 0x7f3cf4160770 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf1d9b640 1 -- 172.21.15.79:0/2254783306 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce8032440 con 0x7f3cf4160770 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf1d9b640 1 -- 172.21.15.79:0/2254783306 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8032810 con 0x7f3cf4160770 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf9140640 1 -- 172.21.15.79:0/2254783306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 msgr2=0x7f3cf40b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf9140640 1 --2- 172.21.15.79:0/2254783306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 0x7f3cf40b4050 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40b5690 tx=0x7f3ce802a1a0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf9140640 1 -- 172.21.15.79:0/2254783306 shutdown_connections 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf9140640 1 --2- 172.21.15.79:0/2254783306 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 0x7f3cf40b4050 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf9140640 1 --2- 172.21.15.79:0/2254783306 >> v2:172.21.15.83:3300/0 conn(0x7f3cf4160390 0x7f3cf40b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.496+0000 7f3cf9140640 1 -- 172.21.15.79:0/2254783306 >> 172.21.15.79:0/2254783306 conn(0x7f3cf40b2720 msgr2=0x7f3cf4070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.497+0000 7f3cf9140640 1 -- 172.21.15.79:0/2254783306 shutdown_connections 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.497+0000 7f3cf9140640 1 -- 172.21.15.79:0/2254783306 wait complete. 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.498+0000 7f3cf9140640 1 Processor -- start 2024-04-26T19:51:40.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.498+0000 7f3cf9140640 1 -- start start 2024-04-26T19:51:40.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf9140640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf4160390 0x7f3cf415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf9140640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 0x7f3cf413a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf9140640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cf41630d0 0x7f3cf413bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf9140640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4145b90 con 0x7f3cf4160770 2024-04-26T19:51:40.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf9140640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cf4145a10 con 0x7f3cf4160390 2024-04-26T19:51:40.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf9140640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3cf4169460 con 0x7f3cf41630d0 2024-04-26T19:51:40.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 0x7f3cf413a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 0x7f3cf413a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53938/0 (socket says 172.21.15.79:53938) 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf259c640 1 -- 172.21.15.79:0/3981561341 learned_addr learned my addr 172.21.15.79:0/3981561341 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf259c640 1 -- 172.21.15.79:0/3981561341 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cf41630d0 msgr2=0x7f3cf413bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf359e640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cf41630d0 0x7f3cf413bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf2d9d640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf4160390 0x7f3cf415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf259c640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cf41630d0 0x7f3cf413bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf259c640 1 -- 172.21.15.79:0/3981561341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf4160390 msgr2=0x7f3cf415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf259c640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf4160390 0x7f3cf415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf259c640 1 -- 172.21.15.79:0/3981561341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cf4148910 con 0x7f3cf4160770 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf259c640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 0x7f3cf413a030 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000c00 tx=0x7f3ce80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:40.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cdb7fe640 1 -- 172.21.15.79:0/3981561341 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8041020 con 0x7f3cf4160770 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cdb7fe640 1 -- 172.21.15.79:0/3981561341 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ce8030970 con 0x7f3cf4160770 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cdb7fe640 1 -- 172.21.15.79:0/3981561341 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8037040 con 0x7f3cf4160770 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cf413b220 con 0x7f3cf4160770 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.499+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cf413b7e0 con 0x7f3cf4160770 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.501+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cf40506c0 con 0x7f3cf4160770 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.501+0000 7f3cdb7fe640 1 -- 172.21.15.79:0/3981561341 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ce8044020 con 0x7f3cf4160770 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.501+0000 7f3cdb7fe640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ce006c8f0 0x7f3ce006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.501+0000 7f3cdb7fe640 1 -- 172.21.15.79:0/3981561341 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f3ce80ac400 con 0x7f3cf4160770 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.501+0000 7f3cf2d9d640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ce006c8f0 0x7f3ce006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.502+0000 7f3cf2d9d640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ce006c8f0 0x7f3ce006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00b840 tx=0x7f3cec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:40.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.506+0000 7f3cdb7fe640 1 -- 172.21.15.79:0/3981561341 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ce804b030 con 0x7f3cf4160770 2024-04-26T19:51:40.682 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.681+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cf40b2500 con 0x7f3cf4160770 2024-04-26T19:51:40.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.682+0000 7f3cdb7fe640 1 -- 172.21.15.79:0/3981561341 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13116 (secure 0 0 0) 0x7f3ce807a9f0 con 0x7f3cf4160770 2024-04-26T19:51:40.683 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:40.683 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":35,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:37.807004+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:40.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ce006c8f0 msgr2=0x7f3ce006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ce006c8f0 0x7f3ce006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00b840 tx=0x7f3cec008040 comp rx=0 tx=0).stop 2024-04-26T19:51:40.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 msgr2=0x7f3cf413a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 0x7f3cf413a030 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8000c00 tx=0x7f3ce80046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 shutdown_connections 2024-04-26T19:51:40.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3ce006c8f0 0x7f3ce006edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cf41630d0 0x7f3cf413bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.685 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3cf4160770 0x7f3cf413a030 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 --2- 172.21.15.79:0/3981561341 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cf4160390 0x7f3cf415d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 >> 172.21.15.79:0/3981561341 conn(0x7f3cf40b2720 msgr2=0x7f3cf4161610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:40.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 shutdown_connections 2024-04-26T19:51:40.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.684+0000 7f3cf9140640 1 -- 172.21.15.79:0/3981561341 wait complete. 2024-04-26T19:51:40.696 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T19:51:40.696 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:51:40.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.794+0000 7f77d0967640 1 Processor -- start 2024-04-26T19:51:40.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.794+0000 7f77d0967640 1 -- start start 2024-04-26T19:51:40.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.795+0000 7f77d0967640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 0x7f77cc0b3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.795+0000 7f77d0967640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77cc06d790 0x7f77cc0b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.795+0000 7f77d0967640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f77cc0b1e40 0x7f77cc0b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.795+0000 7f77d0967640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f77cc162f10 con 0x7f77cc0b1e40 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.795+0000 7f77d0967640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f77cc163120 con 0x7f77cc06d790 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.795+0000 7f77d0967640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f77cc163370 con 0x7f77cc162b30 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.795+0000 7f77cad9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f77cc0b1e40 0x7f77cc0b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.795+0000 7f77cad9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f77cc0b1e40 0x7f77cc0b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53942/0 (socket says 172.21.15.79:53942) 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.795+0000 7f77cad9d640 1 -- 172.21.15.79:0/1409222687 learned_addr learned my addr 172.21.15.79:0/1409222687 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.796+0000 7f77c9d9b640 1 --2- 172.21.15.79:0/1409222687 >> v2:172.21.15.83:3300/0 conn(0x7f77cc06d790 0x7f77cc0b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.796+0000 7f77ca59c640 1 --2- 172.21.15.79:0/1409222687 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 0x7f77cc0b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.796+0000 7f77cad9d640 1 -- 172.21.15.79:0/1409222687 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 msgr2=0x7f77cc0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.796+0000 7f77cad9d640 1 --2- 172.21.15.79:0/1409222687 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 0x7f77cc0b3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.796+0000 7f77cad9d640 1 -- 172.21.15.79:0/1409222687 >> v2:172.21.15.83:3300/0 conn(0x7f77cc06d790 msgr2=0x7f77cc0b39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.796+0000 7f77cad9d640 1 --2- 172.21.15.79:0/1409222687 >> v2:172.21.15.83:3300/0 conn(0x7f77cc06d790 0x7f77cc0b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.796+0000 7f77cad9d640 1 -- 172.21.15.79:0/1409222687 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc163650 con 0x7f77cc0b1e40 2024-04-26T19:51:40.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.796+0000 7f77cad9d640 1 --2- 172.21.15.79:0/1409222687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 0x7f77cc0b64c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f77cc076800 tx=0x7f77c402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ebb5b6b221423e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:40.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.796+0000 7f77c959a640 1 -- 172.21.15.79:0/1409222687 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4037070 con 0x7f77cc0b1e40 2024-04-26T19:51:40.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77c959a640 1 -- 172.21.15.79:0/1409222687 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c402ae00 con 0x7f77cc0b1e40 2024-04-26T19:51:40.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77c959a640 1 -- 172.21.15.79:0/1409222687 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4032660 con 0x7f77cc0b1e40 2024-04-26T19:51:40.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77d0967640 1 -- 172.21.15.79:0/1409222687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 msgr2=0x7f77cc0b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77d0967640 1 --2- 172.21.15.79:0/1409222687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 0x7f77cc0b64c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f77cc076800 tx=0x7f77c402a0b0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77d0967640 1 -- 172.21.15.79:0/1409222687 shutdown_connections 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77d0967640 1 --2- 172.21.15.79:0/1409222687 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 0x7f77cc0b64c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77d0967640 1 --2- 172.21.15.79:0/1409222687 >> v2:172.21.15.83:3300/0 conn(0x7f77cc06d790 0x7f77cc0b39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77d0967640 1 --2- 172.21.15.79:0/1409222687 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 0x7f77cc0b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77d0967640 1 -- 172.21.15.79:0/1409222687 >> 172.21.15.79:0/1409222687 conn(0x7f77cc0b4500 msgr2=0x7f77cc0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77d0967640 1 -- 172.21.15.79:0/1409222687 shutdown_connections 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.797+0000 7f77d0967640 1 -- 172.21.15.79:0/1409222687 wait complete. 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77d0967640 1 Processor -- start 2024-04-26T19:51:40.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77d0967640 1 -- start start 2024-04-26T19:51:40.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77d0967640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77cc06d790 0x7f77cc1f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77d0967640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 0x7f77cc0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77d0967640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 0x7f77cc0cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77d0967640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc0d02f0 con 0x7f77cc0b1e40 2024-04-26T19:51:40.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77d0967640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77cc163c10 con 0x7f77cc06d790 2024-04-26T19:51:40.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77d0967640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f77cc0b1900 con 0x7f77cc162b30 2024-04-26T19:51:40.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77c9d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 0x7f77cc0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77c9d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 0x7f77cc0cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53954/0 (socket says 172.21.15.79:53954) 2024-04-26T19:51:40.803 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.799+0000 7f77c9d9b640 1 -- 172.21.15.79:0/4126504796 learned_addr learned my addr 172.21.15.79:0/4126504796 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77cad9d640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 0x7f77cc0cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77ca59c640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77cc06d790 0x7f77cc1f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77c9d9b640 1 -- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 msgr2=0x7f77cc0cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77c9d9b640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 0x7f77cc0cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77c9d9b640 1 -- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77cc06d790 msgr2=0x7f77cc1f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77c9d9b640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77cc06d790 0x7f77cc1f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77c9d9b640 1 -- 172.21.15.79:0/4126504796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77cc1f5f30 con 0x7f77cc0b1e40 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77cad9d640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 0x7f77cc0cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77ca59c640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77cc06d790 0x7f77cc1f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77c9d9b640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 0x7f77cc0cd750 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f77c000c7f0 tx=0x7f77c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77baffd640 1 -- 172.21.15.79:0/4126504796 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0015070 con 0x7f77cc0b1e40 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77baffd640 1 -- 172.21.15.79:0/4126504796 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77c0004220 con 0x7f77cc0b1e40 2024-04-26T19:51:40.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77baffd640 1 -- 172.21.15.79:0/4126504796 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0007620 con 0x7f77cc0b1e40 2024-04-26T19:51:40.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77cc1f61a0 con 0x7f77cc0b1e40 2024-04-26T19:51:40.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.800+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77cc1f6a40 con 0x7f77cc0b1e40 2024-04-26T19:51:40.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.801+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77cc1f6590 con 0x7f77cc0b1e40 2024-04-26T19:51:40.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.802+0000 7f77baffd640 1 -- 172.21.15.79:0/4126504796 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77c00079d0 con 0x7f77cc0b1e40 2024-04-26T19:51:40.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.802+0000 7f77baffd640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f77ac06c8f0 0x7f77ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:40.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.802+0000 7f77ca59c640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f77ac06c8f0 0x7f77ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:40.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.802+0000 7f77ca59c640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f77ac06c8f0 0x7f77ac06edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77bc007fc0 tx=0x7f77bc0075b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:40.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.805+0000 7f77baffd640 1 -- 172.21.15.79:0/4126504796 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f77c008a1f0 con 0x7f77cc0b1e40 2024-04-26T19:51:40.810 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.810+0000 7f77baffd640 1 -- 172.21.15.79:0/4126504796 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f77c0059fe0 con 0x7f77cc0b1e40 2024-04-26T19:51:40.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.983+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77cc0b9170 con 0x7f77cc0b1e40 2024-04-26T19:51:40.984 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.983+0000 7f77baffd640 1 -- 172.21.15.79:0/4126504796 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f77c005ee20 con 0x7f77cc0b1e40 2024-04-26T19:51:40.984 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:51:40.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f77ac06c8f0 msgr2=0x7f77ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f77ac06c8f0 0x7f77ac06edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f77bc007fc0 tx=0x7f77bc0075b0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 msgr2=0x7f77cc0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:40.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 0x7f77cc0cd750 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f77c000c7f0 tx=0x7f77c000ccb0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.986 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 shutdown_connections 2024-04-26T19:51:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f77ac06c8f0 0x7f77ac06edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f77cc162b30 0x7f77cc0cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77cc0b1e40 0x7f77cc0cd750 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 --2- 172.21.15.79:0/4126504796 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f77cc06d790 0x7f77cc1f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 >> 172.21.15.79:0/4126504796 conn(0x7f77cc0b4500 msgr2=0x7f77cc06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 shutdown_connections 2024-04-26T19:51:40.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:40.985+0000 7f77d0967640 1 -- 172.21.15.79:0/4126504796 wait complete. 2024-04-26T19:51:40.997 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-26T19:51:41.093 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.093+0000 7f3e714b0640 1 Processor -- start 2024-04-26T19:51:41.094 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.093+0000 7f3e714b0640 1 -- start start 2024-04-26T19:51:41.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e714b0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c06ff00 0x7f3e6c148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e714b0640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e6c0d2cd0 0x7f3e6c0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e714b0640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e6c0b1290 con 0x7f3e6c06fb20 2024-04-26T19:51:41.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e714b0640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3e6c0d3e60 con 0x7f3e6c0d2cd0 2024-04-26T19:51:41.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e714b0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6c0d3110 con 0x7f3e6c06ff00 2024-04-26T19:51:41.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e6affd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3e6c06fb20 0x7f3e6c1488c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57202/0 (socket says 172.21.15.79:57202) 2024-04-26T19:51:41.098 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e6affd640 1 -- 172.21.15.79:0/1117846552 learned_addr learned my addr 172.21.15.79:0/1117846552 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e6b7fe640 1 --2- 172.21.15.79:0/1117846552 >> v2:172.21.15.83:3300/0 conn(0x7f3e6c0d2cd0 0x7f3e6c0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e6a7fc640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c06ff00 0x7f3e6c148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3080452913 0 0) 0x7f3e6c0b1290 con 0x7f3e6c06fb20 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3e54003520 con 0x7f3e6c06fb20 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1358127029 0 0) 0x7f3e54003520 con 0x7f3e6c06fb20 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c06ff00 msgr2=0x7f3e6c148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c06ff00 0x7f3e6c148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 >> v2:172.21.15.83:3300/0 conn(0x7f3e6c0d2cd0 msgr2=0x7f3e6c0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 --2- 172.21.15.79:0/1117846552 >> v2:172.21.15.83:3300/0 conn(0x7f3e6c0d2cd0 0x7f3e6c0d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c0d4090 con 0x7f3e6c06fb20 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f3e5c001670 con 0x7f3e6c06fb20 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 >> v1:172.21.15.79:6789/0 conn(0x7f3e6c06fb20 legacy=0x7f3e6c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e540039b0 0x7f3e54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.099 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e54004800 0x7f3e54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c06ff00 0x7f3e540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c0d3110 con 0x7f3e54004800 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e5400b9c0 con 0x7f3e540039b0 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3e5400bb70 con 0x7f3e6c06ff00 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.094+0000 7f3e6a7fc640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e54004800 0x7f3e54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6b7fe640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c06ff00 0x7f3e540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6affd640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e540039b0 0x7f3e54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6a7fc640 1 -- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c06ff00 msgr2=0x7f3e540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6a7fc640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c06ff00 0x7f3e540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6a7fc640 1 -- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e540039b0 msgr2=0x7f3e54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6a7fc640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e540039b0 0x7f3e54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6a7fc640 1 -- 172.21.15.79:0/1117846552 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e54003520 con 0x7f3e54004800 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6b7fe640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c06ff00 0x7f3e540051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:41.100 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6affd640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e540039b0 0x7f3e54003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e6a7fc640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e54004800 0x7f3e54004c80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009ac0 tx=0x7f3e54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230494dff7824ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.095+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60036070 con 0x7f3e54004800 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.096+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e60031400 con 0x7f3e54004800 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.096+0000 7f3e69ffb640 1 -- 172.21.15.79:0/1117846552 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e60009690 con 0x7f3e54004800 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.096+0000 7f3e714b0640 1 -- 172.21.15.79:0/1117846552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e54004800 msgr2=0x7f3e54004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.096+0000 7f3e714b0640 1 --2- 172.21.15.79:0/1117846552 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e54004800 0x7f3e54004c80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3e60009ac0 tx=0x7f3e54005830 comp rx=0 tx=0).stop 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.096+0000 7f3e6b7fe640 1 -- 172.21.15.79:0/1117846552 reap_dead start 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.096+0000 7f3e714b0640 1 -- 172.21.15.79:0/1117846552 shutdown_connections 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.096+0000 7f3e714b0640 1 -- 172.21.15.79:0/1117846552 >> 172.21.15.79:0/1117846552 conn(0x7f3e6c071c30 msgr2=0x7f3e6c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.096+0000 7f3e714b0640 1 -- 172.21.15.79:0/1117846552 shutdown_connections 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.096+0000 7f3e714b0640 1 -- 172.21.15.79:0/1117846552 wait complete. 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.097+0000 7f3e714b0640 1 Processor -- start 2024-04-26T19:51:41.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.097+0000 7f3e714b0640 1 -- start start 2024-04-26T19:51:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e714b0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e54004800 0x7f3e6c162100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e714b0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c162640 0x7f3e6c15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e714b0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c1600a0 0x7f3e6c1f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e714b0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c0d4930 con 0x7f3e6c1600a0 2024-04-26T19:51:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e714b0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c0d4650 con 0x7f3e6c162640 2024-04-26T19:51:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e714b0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6c0d4c10 con 0x7f3e54004800 2024-04-26T19:51:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c1600a0 0x7f3e6c1f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c1600a0 0x7f3e6c1f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53986/0 (socket says 172.21.15.79:53986) 2024-04-26T19:51:41.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6b7fe640 1 -- 172.21.15.79:0/1494482373 learned_addr learned my addr 172.21.15.79:0/1494482373 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6b7fe640 1 -- 172.21.15.79:0/1494482373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e54004800 msgr2=0x7f3e6c162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6a7fc640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c162640 0x7f3e6c15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6affd640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e54004800 0x7f3e6c162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6b7fe640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e54004800 0x7f3e6c162100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6b7fe640 1 -- 172.21.15.79:0/1494482373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c162640 msgr2=0x7f3e6c15fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6b7fe640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c162640 0x7f3e6c15fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6b7fe640 1 -- 172.21.15.79:0/1494482373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c1f86b0 con 0x7f3e6c1600a0 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.098+0000 7f3e6a7fc640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c162640 0x7f3e6c15fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.099+0000 7f3e6b7fe640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c1600a0 0x7f3e6c1f5f50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c149540 tx=0x7f3e64010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.099+0000 7f3e537fe640 1 -- 172.21.15.79:0/1494482373 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64010da0 con 0x7f3e6c1600a0 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.099+0000 7f3e537fe640 1 -- 172.21.15.79:0/1494482373 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e64003990 con 0x7f3e6c1600a0 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.099+0000 7f3e537fe640 1 -- 172.21.15.79:0/1494482373 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64003d80 con 0x7f3e6c1600a0 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.099+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6c160740 con 0x7f3e6c1600a0 2024-04-26T19:51:41.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.099+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6c160d00 con 0x7f3e6c1600a0 2024-04-26T19:51:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.100+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e6c0506c0 con 0x7f3e6c1600a0 2024-04-26T19:51:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.100+0000 7f3e537fe640 1 -- 172.21.15.79:0/1494482373 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3e64005d10 con 0x7f3e6c1600a0 2024-04-26T19:51:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.101+0000 7f3e537fe640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3e5806c8f0 0x7f3e5806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.101+0000 7f3e6affd640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3e5806c8f0 0x7f3e5806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.101+0000 7f3e537fe640 1 -- 172.21.15.79:0/1494482373 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f3e6408c130 con 0x7f3e6c1600a0 2024-04-26T19:51:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.101+0000 7f3e6affd640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3e5806c8f0 0x7f3e5806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c001eb0 tx=0x7f3e5c000ed0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:41.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.105+0000 7f3e537fe640 1 -- 172.21.15.79:0/1494482373 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e64025020 con 0x7f3e6c1600a0 2024-04-26T19:51:41.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.281+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3e6c162a20 con 0x7f3e6c1600a0 2024-04-26T19:51:41.282 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.282+0000 7f3e537fe640 1 -- 172.21.15.79:0/1494482373 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f3e6c1f86b0 con 0x7f3e6c1600a0 2024-04-26T19:51:41.282 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:41.282 INFO:teuthology.orchestra.run.smithi079.stdout:[ 2024-04-26T19:51:41.282 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:41.283 INFO:teuthology.orchestra.run.smithi079.stdout: "poolnum": 1, 2024-04-26T19:51:41.283 INFO:teuthology.orchestra.run.smithi079.stdout: "poolname": ".mgr" 2024-04-26T19:51:41.283 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:41.283 INFO:teuthology.orchestra.run.smithi079.stdout:] 2024-04-26T19:51:41.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3e5806c8f0 msgr2=0x7f3e5806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3e5806c8f0 0x7f3e5806edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c001eb0 tx=0x7f3e5c000ed0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c1600a0 msgr2=0x7f3e6c1f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c1600a0 0x7f3e6c1f5f50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c149540 tx=0x7f3e64010590 comp rx=0 tx=0).stop 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 shutdown_connections 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f3e5806c8f0 0x7f3e5806edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c1600a0 0x7f3e6c1f5f50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c162640 0x7f3e6c15fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 --2- 172.21.15.79:0/1494482373 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e54004800 0x7f3e6c162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 >> 172.21.15.79:0/1494482373 conn(0x7f3e6c071c30 msgr2=0x7f3e6c144600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 shutdown_connections 2024-04-26T19:51:41.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.284+0000 7f3e714b0640 1 -- 172.21.15.79:0/1494482373 wait complete. 2024-04-26T19:51:41.295 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T19:51:41.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T19:51:41.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.395+0000 7f432c940640 1 Processor -- start 2024-04-26T19:51:41.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.395+0000 7f432c940640 1 -- start start 2024-04-26T19:51:41.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f432c940640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f432806ff50 0x7f43280cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f432c940640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c460 0x7f432806df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f432c940640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43280cf280 con 0x7f432815c080 2024-04-26T19:51:41.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f432c940640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f43280cd7d0 con 0x7f432806ff50 2024-04-26T19:51:41.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f432c940640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f43280cf540 con 0x7f432815c460 2024-04-26T19:51:41.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f431ffff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f432806ff50 0x7f43280cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f432659c640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f432815c080 0x7f432806f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57208/0 (socket says 172.21.15.79:57208) 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f431ffff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f432806ff50 0x7f43280cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:53998/0 (socket says 172.21.15.79:53998) 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f432659c640 1 -- 172.21.15.79:0/3653929949 learned_addr learned my addr 172.21.15.79:0/3653929949 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f4326d9d640 1 --2- 172.21.15.79:0/3653929949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c460 0x7f432806df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f431ffff640 1 -- 172.21.15.79:0/3653929949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c460 msgr2=0x7f432806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f431ffff640 1 --2- 172.21.15.79:0/3653929949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c460 0x7f432806df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f431ffff640 1 -- 172.21.15.79:0/3653929949 >> v1:172.21.15.79:6789/0 conn(0x7f432815c080 legacy=0x7f432806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f431ffff640 1 -- 172.21.15.79:0/3653929949 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43280cf820 con 0x7f432806ff50 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f4325d9b640 1 -- 172.21.15.79:0/3653929949 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3069237921 0 0) 0x7f43280cf280 con 0x7f432815c080 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.396+0000 7f4326d9d640 1 --2- 172.21.15.79:0/3653929949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c460 0x7f432806df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.397+0000 7f431ffff640 1 --2- 172.21.15.79:0/3653929949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432806ff50 0x7f43280cfd40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f43100097a0 tx=0x7f431002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1cfcb76bb3b37a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.397+0000 7f4325d9b640 1 -- 172.21.15.79:0/3653929949 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310037070 con 0x7f432806ff50 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.397+0000 7f4325d9b640 1 -- 172.21.15.79:0/3653929949 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f431002ae50 con 0x7f432806ff50 2024-04-26T19:51:41.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.397+0000 7f4325d9b640 1 -- 172.21.15.79:0/3653929949 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310032660 con 0x7f432806ff50 2024-04-26T19:51:41.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.398+0000 7f432c940640 1 -- 172.21.15.79:0/3653929949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432806ff50 msgr2=0x7f43280cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.398+0000 7f432c940640 1 --2- 172.21.15.79:0/3653929949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432806ff50 0x7f43280cfd40 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f43100097a0 tx=0x7f431002a100 comp rx=0 tx=0).stop 2024-04-26T19:51:41.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.398+0000 7f432c940640 1 -- 172.21.15.79:0/3653929949 shutdown_connections 2024-04-26T19:51:41.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.398+0000 7f432c940640 1 --2- 172.21.15.79:0/3653929949 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c460 0x7f432806df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.398+0000 7f432c940640 1 --2- 172.21.15.79:0/3653929949 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432806ff50 0x7f43280cfd40 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.398+0000 7f432c940640 1 -- 172.21.15.79:0/3653929949 >> 172.21.15.79:0/3653929949 conn(0x7f4328068840 msgr2=0x7f43280d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:41.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.398+0000 7f432c940640 1 -- 172.21.15.79:0/3653929949 shutdown_connections 2024-04-26T19:51:41.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.398+0000 7f432c940640 1 -- 172.21.15.79:0/3653929949 wait complete. 2024-04-26T19:51:41.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.399+0000 7f432c940640 1 Processor -- start 2024-04-26T19:51:41.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.399+0000 7f432c940640 1 -- start start 2024-04-26T19:51:41.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432c940640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f432806ff50 0x7f43280cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432c940640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c080 0x7f43280ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432c940640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432815c460 0x7f4328153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432c940640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f43280b01b0 con 0x7f432815c460 2024-04-26T19:51:41.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432c940640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f43280b0030 con 0x7f432806ff50 2024-04-26T19:51:41.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432c940640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f43280b0330 con 0x7f432815c080 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f4326d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432815c460 0x7f4328153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f4326d9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432815c460 0x7f4328153190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54014/0 (socket says 172.21.15.79:54014) 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f4326d9d640 1 -- 172.21.15.79:0/1644712116 learned_addr learned my addr 172.21.15.79:0/1644712116 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f431ffff640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c080 0x7f43280ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432659c640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f432806ff50 0x7f43280cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f4326d9d640 1 -- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c080 msgr2=0x7f43280ce240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f4326d9d640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c080 0x7f43280ce240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f4326d9d640 1 -- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f432806ff50 msgr2=0x7f43280cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f4326d9d640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f432806ff50 0x7f43280cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f4326d9d640 1 -- 172.21.15.79:0/1644712116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4328153ec0 con 0x7f432815c460 2024-04-26T19:51:41.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432659c640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f432806ff50 0x7f43280cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:41.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f431ffff640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c080 0x7f43280ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:41.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f4326d9d640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432815c460 0x7f4328153190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f432000cd50 tx=0x7f432000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:41.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f431effd640 1 -- 172.21.15.79:0/1644712116 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432000f680 con 0x7f432815c460 2024-04-26T19:51:41.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f431effd640 1 -- 172.21.15.79:0/1644712116 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432000f840 con 0x7f432815c460 2024-04-26T19:51:41.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4328154500 con 0x7f432815c460 2024-04-26T19:51:41.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f431effd640 1 -- 172.21.15.79:0/1644712116 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432000fc30 con 0x7f432815c460 2024-04-26T19:51:41.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.400+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4328154ac0 con 0x7f432815c460 2024-04-26T19:51:41.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.402+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4328154cc0 con 0x7f432815c460 2024-04-26T19:51:41.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.402+0000 7f431effd640 1 -- 172.21.15.79:0/1644712116 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43200213f0 con 0x7f432815c460 2024-04-26T19:51:41.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.402+0000 7f431effd640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f430806c8f0 0x7f430806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:41.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.403+0000 7f431effd640 1 -- 172.21.15.79:0/1644712116 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f432008c830 con 0x7f432815c460 2024-04-26T19:51:41.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.403+0000 7f432659c640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f430806c8f0 0x7f430806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:41.407 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.403+0000 7f432659c640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f430806c8f0 0x7f430806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4318000f60 tx=0x7f4318007c30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:41.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.407+0000 7f431effd640 1 -- 172.21.15.79:0/1644712116 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4328154cc0 con 0x7f432815c460 2024-04-26T19:51:41.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:41.579+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f43280506c0 con 0x7f432815c460 2024-04-26T19:51:42.022 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.021+0000 7f431effd640 1 -- 172.21.15.79:0/1644712116 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7f4320014030 con 0x7f432815c460 2024-04-26T19:51:42.022 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' created 2024-04-26T19:51:42.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f430806c8f0 msgr2=0x7f430806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:42.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f430806c8f0 0x7f430806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4318000f60 tx=0x7f4318007c30 comp rx=0 tx=0).stop 2024-04-26T19:51:42.024 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432815c460 msgr2=0x7f4328153190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:42.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432815c460 0x7f4328153190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f432000cd50 tx=0x7f432000a650 comp rx=0 tx=0).stop 2024-04-26T19:51:42.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 shutdown_connections 2024-04-26T19:51:42.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f430806c8f0 0x7f430806edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f432815c460 0x7f4328153190 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f432815c080 0x7f43280ce240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 --2- 172.21.15.79:0/1644712116 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f432806ff50 0x7f43280cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.023+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 >> 172.21.15.79:0/1644712116 conn(0x7f4328068840 msgr2=0x7f43280b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:42.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.024+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 shutdown_connections 2024-04-26T19:51:42.025 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.024+0000 7f432c940640 1 -- 172.21.15.79:0/1644712116 wait complete. 2024-04-26T19:51:42.039 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T19:51:42.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.145+0000 7f943f807640 1 Processor -- start 2024-04-26T19:51:42.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.145+0000 7f943f807640 1 -- start start 2024-04-26T19:51:42.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943f807640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9438156670 0x7f943815d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:42.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943f807640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f94380cd9c0 0x7f94380b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:42.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943f807640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 0x7f94380b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943f807640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f9438073110 con 0x7f94380cd9c0 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943f807640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f943815d670 con 0x7f9438156670 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943f807640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f94380ce1a0 con 0x7f94380735a0 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943cda2640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f94380cd9c0 0x7f94380b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943cda2640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f94380cd9c0 0x7f94380b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54028/0 (socket says 172.21.15.79:54028) 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943cda2640 1 -- 172.21.15.79:0/1633393163 learned_addr learned my addr 172.21.15.79:0/1633393163 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943dda4640 1 --2- 172.21.15.79:0/1633393163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 0x7f94380b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943d5a3640 1 --2- 172.21.15.79:0/1633393163 >> v2:172.21.15.83:3300/0 conn(0x7f9438156670 0x7f943815d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943cda2640 1 -- 172.21.15.79:0/1633393163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 msgr2=0x7f94380b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943cda2640 1 --2- 172.21.15.79:0/1633393163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 0x7f94380b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943cda2640 1 -- 172.21.15.79:0/1633393163 >> v2:172.21.15.83:3300/0 conn(0x7f9438156670 msgr2=0x7f943815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943cda2640 1 --2- 172.21.15.79:0/1633393163 >> v2:172.21.15.83:3300/0 conn(0x7f9438156670 0x7f943815d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.146+0000 7f943cda2640 1 -- 172.21.15.79:0/1633393163 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94380cdec0 con 0x7f94380cd9c0 2024-04-26T19:51:42.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.147+0000 7f943cda2640 1 --2- 172.21.15.79:0/1633393163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94380cd9c0 0x7f94380b3de0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9438073fa0 tx=0x7f942802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8759713b64ce8414 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.147+0000 7f942ffff640 1 -- 172.21.15.79:0/1633393163 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428037070 con 0x7f94380cd9c0 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.147+0000 7f942ffff640 1 -- 172.21.15.79:0/1633393163 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9428032440 con 0x7f94380cd9c0 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.147+0000 7f942ffff640 1 -- 172.21.15.79:0/1633393163 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9428032810 con 0x7f94380cd9c0 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.148+0000 7f943f807640 1 -- 172.21.15.79:0/1633393163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94380cd9c0 msgr2=0x7f94380b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.148+0000 7f943f807640 1 --2- 172.21.15.79:0/1633393163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94380cd9c0 0x7f94380b3de0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9438073fa0 tx=0x7f942802a1e0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.148+0000 7f943f807640 1 -- 172.21.15.79:0/1633393163 shutdown_connections 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.148+0000 7f943f807640 1 --2- 172.21.15.79:0/1633393163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 0x7f94380b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.148+0000 7f943f807640 1 --2- 172.21.15.79:0/1633393163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f94380cd9c0 0x7f94380b3de0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.148+0000 7f943f807640 1 --2- 172.21.15.79:0/1633393163 >> v2:172.21.15.83:3300/0 conn(0x7f9438156670 0x7f943815d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.148+0000 7f943f807640 1 -- 172.21.15.79:0/1633393163 >> 172.21.15.79:0/1633393163 conn(0x7f94380b39d0 msgr2=0x7f943806feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.148+0000 7f943f807640 1 -- 172.21.15.79:0/1633393163 shutdown_connections 2024-04-26T19:51:42.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.148+0000 7f943f807640 1 -- 172.21.15.79:0/1633393163 wait complete. 2024-04-26T19:51:42.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.149+0000 7f943f807640 1 Processor -- start 2024-04-26T19:51:42.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.149+0000 7f943f807640 1 -- start start 2024-04-26T19:51:42.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943f807640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 0x7f9438152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:42.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943f807640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94380cd9c0 0x7f94381efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:42.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943f807640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9438156670 0x7f94381f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:42.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943f807640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9438156a50 con 0x7f9438156670 2024-04-26T19:51:42.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943f807640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9438072da0 con 0x7f94380cd9c0 2024-04-26T19:51:42.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943f807640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9438156bd0 con 0x7f94380735a0 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943dda4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9438156670 0x7f94381f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943dda4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9438156670 0x7f94381f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54034/0 (socket says 172.21.15.79:54034) 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943dda4640 1 -- 172.21.15.79:0/809408577 learned_addr learned my addr 172.21.15.79:0/809408577 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943d5a3640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 0x7f9438152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943cda2640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94380cd9c0 0x7f94381efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943dda4640 1 -- 172.21.15.79:0/809408577 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 msgr2=0x7f9438152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943dda4640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 0x7f9438152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943dda4640 1 -- 172.21.15.79:0/809408577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94380cd9c0 msgr2=0x7f94381efdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943dda4640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94380cd9c0 0x7f94381efdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943dda4640 1 -- 172.21.15.79:0/809408577 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94381f0830 con 0x7f9438156670 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f943dda4640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9438156670 0x7f94381f02f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f943000d800 tx=0x7f943000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.150+0000 7f942dffb640 1 -- 172.21.15.79:0/809408577 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430018020 con 0x7f9438156670 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.151+0000 7f942dffb640 1 -- 172.21.15.79:0/809408577 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94300096c0 con 0x7f9438156670 2024-04-26T19:51:42.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.151+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9438152770 con 0x7f9438156670 2024-04-26T19:51:42.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.151+0000 7f942dffb640 1 -- 172.21.15.79:0/809408577 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9430009a90 con 0x7f9438156670 2024-04-26T19:51:42.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.151+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94380b1580 con 0x7f9438156670 2024-04-26T19:51:42.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.152+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94380b1d50 con 0x7f9438156670 2024-04-26T19:51:42.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.152+0000 7f942dffb640 1 -- 172.21.15.79:0/809408577 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f943001b020 con 0x7f9438156670 2024-04-26T19:51:42.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.152+0000 7f942dffb640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f941c06c8f0 0x7f941c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:42.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.153+0000 7f942dffb640 1 -- 172.21.15.79:0/809408577 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7f943008bdf0 con 0x7f9438156670 2024-04-26T19:51:42.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.153+0000 7f943d5a3640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f941c06c8f0 0x7f941c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:42.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.153+0000 7f943d5a3640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f941c06c8f0 0x7f941c06edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9434003aa0 tx=0x7f9434009830 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:42.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.157+0000 7f942dffb640 1 -- 172.21.15.79:0/809408577 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9430022020 con 0x7f9438156670 2024-04-26T19:51:42.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:42.335+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f94380506c0 con 0x7f9438156670 2024-04-26T19:51:43.028 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.028+0000 7f942dffb640 1 -- 172.21.15.79:0/809408577 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7f9430059cb0 con 0x7f9438156670 2024-04-26T19:51:43.029 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' created 2024-04-26T19:51:43.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f941c06c8f0 msgr2=0x7f941c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:43.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f941c06c8f0 0x7f941c06edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9434003aa0 tx=0x7f9434009830 comp rx=0 tx=0).stop 2024-04-26T19:51:43.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9438156670 msgr2=0x7f94381f02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:43.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9438156670 0x7f94381f02f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f943000d800 tx=0x7f943000dcc0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 shutdown_connections 2024-04-26T19:51:43.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f941c06c8f0 0x7f941c06edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.031 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9438156670 0x7f94381f02f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f94380cd9c0 0x7f94381efdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 --2- 172.21.15.79:0/809408577 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f94380735a0 0x7f9438152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 >> 172.21.15.79:0/809408577 conn(0x7f94380b39d0 msgr2=0x7f94380d0090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:43.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.030+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 shutdown_connections 2024-04-26T19:51:43.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.031+0000 7f943f807640 1 -- 172.21.15.79:0/809408577 wait complete. 2024-04-26T19:51:43.051 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T19:51:43.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.151+0000 7f71df045640 1 Processor -- start 2024-04-26T19:51:43.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.151+0000 7f71df045640 1 -- start start 2024-04-26T19:51:43.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.152+0000 7f71df045640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d81435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.152+0000 7f71df045640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71d80d2d80 0x7f71d80d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.152+0000 7f71df045640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71d8161370 con 0x7f71d80cffe0 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.152+0000 7f71df045640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f71d8161120 con 0x7f71d80d2d80 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.152+0000 7f71df045640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f71d80d0960 con 0x7f71d8140480 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71dcde1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d81435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71dd5e2640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f71d80d2d80 0x7f71d80d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71dcde1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d81435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:52816/0 (socket says 172.21.15.79:52816) 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71dcde1640 1 -- 172.21.15.79:0/3405093616 learned_addr learned my addr 172.21.15.79:0/3405093616 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71dd5e2640 1 -- 172.21.15.79:0/3405093616 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 msgr2=0x7f71d81435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71dd5e2640 1 --2- 172.21.15.79:0/3405093616 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d81435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71dd5e2640 1 -- 172.21.15.79:0/3405093616 >> v1:172.21.15.83:6789/0 conn(0x7f71d80cffe0 legacy=0x7f71d806f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:43.154 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71dd5e2640 1 -- 172.21.15.79:0/3405093616 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d8161650 con 0x7f71d80d2d80 2024-04-26T19:51:43.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71d77fe640 1 -- 172.21.15.79:0/3405093616 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261316152 0 0) 0x7f71d8161370 con 0x7f71d80cffe0 2024-04-26T19:51:43.155 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.153+0000 7f71dcde1640 1 --2- 172.21.15.79:0/3405093616 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d81435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.154+0000 7f71dd5e2640 1 --2- 172.21.15.79:0/3405093616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 0x7f71d80d0420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009d90 tx=0x7f71d002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a451bfa154cb984 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.154+0000 7f71d77fe640 1 -- 172.21.15.79:0/3405093616 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0007740 con 0x7f71d80d2d80 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.154+0000 7f71d77fe640 1 -- 172.21.15.79:0/3405093616 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d00078e0 con 0x7f71d80d2d80 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.154+0000 7f71d77fe640 1 -- 172.21.15.79:0/3405093616 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d0007cb0 con 0x7f71d80d2d80 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.155+0000 7f71df045640 1 -- 172.21.15.79:0/3405093616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 msgr2=0x7f71d80d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.155+0000 7f71df045640 1 --2- 172.21.15.79:0/3405093616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 0x7f71d80d0420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f71d0009d90 tx=0x7f71d002a660 comp rx=0 tx=0).stop 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.155+0000 7f71df045640 1 -- 172.21.15.79:0/3405093616 shutdown_connections 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.155+0000 7f71df045640 1 --2- 172.21.15.79:0/3405093616 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 0x7f71d80d0420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.155+0000 7f71df045640 1 --2- 172.21.15.79:0/3405093616 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d81435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.155+0000 7f71df045640 1 -- 172.21.15.79:0/3405093616 >> 172.21.15.79:0/3405093616 conn(0x7f71d80685d0 msgr2=0x7f71d8073c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:43.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.155+0000 7f71df045640 1 -- 172.21.15.79:0/3405093616 shutdown_connections 2024-04-26T19:51:43.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.155+0000 7f71df045640 1 -- 172.21.15.79:0/3405093616 wait complete. 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.156+0000 7f71df045640 1 Processor -- start 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.156+0000 7f71df045640 1 -- start start 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71df045640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d80cffe0 0x7f71d806e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71df045640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 0x7f71d815db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71df045640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d8156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71df045640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8143b10 con 0x7f71d80cffe0 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71df045640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f71d8161c10 con 0x7f71d80d2d80 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71df045640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f71d8140860 con 0x7f71d8140480 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dcde1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d80cffe0 0x7f71d806e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dcde1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d80cffe0 0x7f71d806e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54048/0 (socket says 172.21.15.79:54048) 2024-04-26T19:51:43.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dcde1640 1 -- 172.21.15.79:0/3995606574 learned_addr learned my addr 172.21.15.79:0/3995606574 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dd5e2640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d8156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71d7fff640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 0x7f71d815db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dcde1640 1 -- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 msgr2=0x7f71d8156290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dcde1640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d8156290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dcde1640 1 -- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 msgr2=0x7f71d815db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dcde1640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 0x7f71d815db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dcde1640 1 -- 172.21.15.79:0/3995606574 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d815e650 con 0x7f71d80cffe0 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dd5e2640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71d7fff640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 0x7f71d815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71dcde1640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d80cffe0 0x7f71d806e7d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e9b0 tx=0x7f71c800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71d57fa640 1 -- 172.21.15.79:0/3995606574 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c8017070 con 0x7f71d80cffe0 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71d57fa640 1 -- 172.21.15.79:0/3995606574 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71c800c9a0 con 0x7f71d80cffe0 2024-04-26T19:51:43.161 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d815e930 con 0x7f71d80cffe0 2024-04-26T19:51:43.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71d57fa640 1 -- 172.21.15.79:0/3995606574 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800cd50 con 0x7f71d80cffe0 2024-04-26T19:51:43.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.157+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d8158720 con 0x7f71d80cffe0 2024-04-26T19:51:43.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.158+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d80ac770 con 0x7f71d80cffe0 2024-04-26T19:51:43.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.159+0000 7f71d57fa640 1 -- 172.21.15.79:0/3995606574 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71c801a070 con 0x7f71d80cffe0 2024-04-26T19:51:43.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.159+0000 7f71d57fa640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f71c406c920 0x7f71c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:43.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.159+0000 7f71d57fa640 1 -- 172.21.15.79:0/3995606574 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f71c808be90 con 0x7f71d80cffe0 2024-04-26T19:51:43.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.159+0000 7f71d7fff640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f71c406c920 0x7f71c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:43.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.160+0000 7f71d7fff640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f71c406c920 0x7f71c406ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f71d81f67f0 tx=0x7f71cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:43.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.164+0000 7f71d57fa640 1 -- 172.21.15.79:0/3995606574 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f71d80ac770 con 0x7f71d80cffe0 2024-04-26T19:51:43.379 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:43.378+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f71d80506c0 con 0x7f71d80cffe0 2024-04-26T19:51:44.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.029+0000 7f71d57fa640 1 -- 172.21.15.79:0/3995606574 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) ==== 172+0+0 (secure 0 0 0) 0x7f71c805a190 con 0x7f71d80cffe0 2024-04-26T19:51:44.030 INFO:teuthology.orchestra.run.smithi079.stderr:new fs with metadata pool 5 and data pool 6 2024-04-26T19:51:44.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.033+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f71c406c920 msgr2=0x7f71c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.034 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.033+0000 7f71df045640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f71c406c920 0x7f71c406ede0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f71d81f67f0 tx=0x7f71cc009040 comp rx=0 tx=0).stop 2024-04-26T19:51:44.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.033+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d80cffe0 msgr2=0x7f71d806e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.035 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.033+0000 7f71df045640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d80cffe0 0x7f71d806e7d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e9b0 tx=0x7f71c800ee70 comp rx=0 tx=0).stop 2024-04-26T19:51:44.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.035+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 shutdown_connections 2024-04-26T19:51:44.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.035+0000 7f71df045640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f71c406c920 0x7f71c406ede0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.035+0000 7f71df045640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f71d8140480 0x7f71d8156290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.035+0000 7f71df045640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f71d80d2d80 0x7f71d815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.035+0000 7f71df045640 1 --2- 172.21.15.79:0/3995606574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f71d80cffe0 0x7f71d806e7d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.035+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 >> 172.21.15.79:0/3995606574 conn(0x7f71d80685d0 msgr2=0x7f71d80ab740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:44.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.036+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 shutdown_connections 2024-04-26T19:51:44.037 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.036+0000 7f71df045640 1 -- 172.21.15.79:0/3995606574 wait complete. 2024-04-26T19:51:44.067 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:44.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.163+0000 7fe3ed9a4640 1 Processor -- start 2024-04-26T19:51:44.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.163+0000 7fe3ed9a4640 1 -- start start 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.164+0000 7fe3ed9a4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3e80b4250 0x7fe3e8163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.164+0000 7fe3ed9a4640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3e8160280 con 0x7fe3e8159890 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.164+0000 7fe3ed9a4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3e8160560 con 0x7fe3e8159c70 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.164+0000 7fe3ed9a4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e8160840 con 0x7fe3e80b4250 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.164+0000 7fe3e6ffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe3e8159890 0x7fe3e815fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57224/0 (socket says 172.21.15.79:57224) 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.164+0000 7fe3e6ffd640 1 -- 172.21.15.79:0/3592986009 learned_addr learned my addr 172.21.15.79:0/3592986009 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453825075 0 0) 0x7fe3e8160280 con 0x7fe3e8159890 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e77fe640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3e80b4250 0x7fe3e8163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3d0003520 con 0x7fe3e8159890 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1287286423 0 0) 0x7fe3d0003520 con 0x7fe3e8159890 2024-04-26T19:51:44.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3e80b4250 msgr2=0x7fe3e8163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3e80b4250 0x7fe3e8163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 >> v1:172.21.15.83:6789/0 conn(0x7fe3e8159c70 legacy=0x7fe3e80b3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e8160e20 con 0x7fe3e8159890 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fe3d80019f0 con 0x7fe3e8159890 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 >> v1:172.21.15.79:6789/0 conn(0x7fe3e8159890 legacy=0x7fe3e815fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00039e0 0x7fe3d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3d0004800 0x7fe3d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3e80b4250 0x7fe3d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8160840 con 0x7fe3d0004800 2024-04-26T19:51:44.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d000baa0 con 0x7fe3d00039e0 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d000bca0 con 0x7fe3e80b4250 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e67fc640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3d0004800 0x7fe3d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e6ffd640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00039e0 0x7fe3d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e77fe640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3e80b4250 0x7fe3d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e67fc640 1 -- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3e80b4250 msgr2=0x7fe3d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e67fc640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3e80b4250 0x7fe3d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e67fc640 1 -- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00039e0 msgr2=0x7fe3d0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e67fc640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3d00039e0 0x7fe3d0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.165+0000 7fe3e67fc640 1 -- 172.21.15.79:0/3592986009 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d0003520 con 0x7fe3d0004800 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.166+0000 7fe3e67fc640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3d0004800 0x7fe3d0004c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001720 tx=0x7fe3d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37de79aefe947631 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.166+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc001370 con 0x7fe3d0004800 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.166+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3dc0052d0 con 0x7fe3d0004800 2024-04-26T19:51:44.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.166+0000 7fe3e5ffb640 1 -- 172.21.15.79:0/3592986009 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc02ab90 con 0x7fe3d0004800 2024-04-26T19:51:44.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.167+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/3592986009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3d0004800 msgr2=0x7fe3d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.167+0000 7fe3ed9a4640 1 --2- 172.21.15.79:0/3592986009 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3d0004800 0x7fe3d0004c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001720 tx=0x7fe3d0005830 comp rx=0 tx=0).stop 2024-04-26T19:51:44.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.167+0000 7fe3e77fe640 1 -- 172.21.15.79:0/3592986009 reap_dead start 2024-04-26T19:51:44.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.167+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/3592986009 shutdown_connections 2024-04-26T19:51:44.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.167+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/3592986009 >> 172.21.15.79:0/3592986009 conn(0x7fe3e80b2720 msgr2=0x7fe3e8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:44.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.167+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/3592986009 shutdown_connections 2024-04-26T19:51:44.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.167+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/3592986009 wait complete. 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.168+0000 7fe3ed9a4640 1 Processor -- start 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.168+0000 7fe3ed9a4640 1 -- start start 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3ed9a4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3d0004800 0x7fe3e815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3ed9a4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e81565e0 0x7fe3e8143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3ed9a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e80b0cc0 0x7fe3e80b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3ed9a4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e81624d0 con 0x7fe3e81565e0 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3ed9a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e8162350 con 0x7fe3e80b0cc0 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3ed9a4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e8162650 con 0x7fe3d0004800 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e67fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e81565e0 0x7fe3e8143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e67fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e81565e0 0x7fe3e8143db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54070/0 (socket says 172.21.15.79:54070) 2024-04-26T19:51:44.174 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e67fc640 1 -- 172.21.15.79:0/4140024163 learned_addr learned my addr 172.21.15.79:0/4140024163 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e77fe640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e80b0cc0 0x7fe3e80b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e6ffd640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3d0004800 0x7fe3e815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e67fc640 1 -- 172.21.15.79:0/4140024163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3d0004800 msgr2=0x7fe3e815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e67fc640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3d0004800 0x7fe3e815a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e67fc640 1 -- 172.21.15.79:0/4140024163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e80b0cc0 msgr2=0x7fe3e80b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e67fc640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e80b0cc0 0x7fe3e80b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e67fc640 1 -- 172.21.15.79:0/4140024163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e81442f0 con 0x7fe3e81565e0 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e6ffd640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3d0004800 0x7fe3e815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3e67fc640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e81565e0 0x7fe3e8143db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001840 tx=0x7fe3dc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3cffff640 1 -- 172.21.15.79:0/4140024163 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc037040 con 0x7fe3e81565e0 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3cffff640 1 -- 172.21.15.79:0/4140024163 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3dc005480 con 0x7fe3e81565e0 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3cffff640 1 -- 172.21.15.79:0/4140024163 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc007a30 con 0x7fe3e81565e0 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.169+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e815a780 con 0x7fe3e81565e0 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.170+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e815ad40 con 0x7fe3e81565e0 2024-04-26T19:51:44.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.170+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e815edd0 con 0x7fe3e81565e0 2024-04-26T19:51:44.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.171+0000 7fe3cffff640 1 -- 172.21.15.79:0/4140024163 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3dc044020 con 0x7fe3e81565e0 2024-04-26T19:51:44.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.171+0000 7fe3cffff640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3d406c8f0 0x7fe3d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.171+0000 7fe3cffff640 1 -- 172.21.15.79:0/4140024163 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe3dc0ab690 con 0x7fe3e81565e0 2024-04-26T19:51:44.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.171+0000 7fe3e6ffd640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3d406c8f0 0x7fe3d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.172+0000 7fe3e6ffd640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3d406c8f0 0x7fe3d406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8004c20 tx=0x7fe3d8006040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:44.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.176+0000 7fe3cffff640 1 -- 172.21.15.79:0/4140024163 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe3e815edd0 con 0x7fe3e81565e0 2024-04-26T19:51:44.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.353+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3e80506c0 con 0x7fe3e81565e0 2024-04-26T19:51:44.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.354+0000 7fe3cffff640 1 -- 172.21.15.79:0/4140024163 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16907 (secure 0 0 0) 0x7fe3dc04b020 con 0x7fe3e81565e0 2024-04-26T19:51:44.355 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:44.355 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":38,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:43.380056+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:51:41.583741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T19:51:42.339687+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3d406c8f0 msgr2=0x7fe3d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3d406c8f0 0x7fe3d406edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8004c20 tx=0x7fe3d8006040 comp rx=0 tx=0).stop 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e81565e0 msgr2=0x7fe3e8143db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e81565e0 0x7fe3e8143db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc001840 tx=0x7fe3dc001120 comp rx=0 tx=0).stop 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 shutdown_connections 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe3d406c8f0 0x7fe3d406edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe3e80b0cc0 0x7fe3e80b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe3e81565e0 0x7fe3e8143db0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 --2- 172.21.15.79:0/4140024163 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe3d0004800 0x7fe3e815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 >> 172.21.15.79:0/4140024163 conn(0x7fe3e80b2720 msgr2=0x7fe3e815d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 shutdown_connections 2024-04-26T19:51:44.357 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.357+0000 7fe3ed9a4640 1 -- 172.21.15.79:0/4140024163 wait complete. 2024-04-26T19:51:44.368 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:44.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff5240da640 1 Processor -- start 2024-04-26T19:51:44.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff5240da640 1 -- start start 2024-04-26T19:51:44.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff5240da640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c0cd9c0 0x7ff51c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff5240da640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff51c0ce1a0 con 0x7ff51c156670 2024-04-26T19:51:44.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff5240da640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff51c0cdec0 con 0x7ff51c0735a0 2024-04-26T19:51:44.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff5240da640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7ff51c073110 con 0x7ff51c0cd9c0 2024-04-26T19:51:44.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff521e76640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7ff51c156670 0x7ff51c15d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57238/0 (socket says 172.21.15.79:57238) 2024-04-26T19:51:44.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff521e76640 1 -- 172.21.15.79:0/409886486 learned_addr learned my addr 172.21.15.79:0/409886486 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:44.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2612349590 0 0) 0x7ff51c0ce1a0 con 0x7ff51c156670 2024-04-26T19:51:44.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff521675640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c0cd9c0 0x7ff51c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.466+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff508003520 con 0x7ff51c156670 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 641526926 0 0) 0x7ff508003520 con 0x7ff51c156670 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c0cd9c0 msgr2=0x7ff51c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c0cd9c0 0x7ff51c0b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 >> v1:172.21.15.83:6789/0 conn(0x7ff51c0735a0 legacy=0x7ff51c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c072ac0 con 0x7ff51c156670 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7ff5180019f0 con 0x7ff51c156670 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 >> v1:172.21.15.79:6789/0 conn(0x7ff51c156670 legacy=0x7ff51c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff508003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c0cd9c0 0x7ff508004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff508004d40 0x7ff5080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c073110 con 0x7ff5080039e0 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff50800ba70 con 0x7ff508004d40 2024-04-26T19:51:44.471 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7ff50800de80 con 0x7ff51c0cd9c0 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff521e76640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff508003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff521675640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c0cd9c0 0x7ff508004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff522677640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff508004d40 0x7ff5080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff521e76640 1 -- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c0cd9c0 msgr2=0x7ff508004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff521e76640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c0cd9c0 0x7ff508004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff521e76640 1 -- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff508004d40 msgr2=0x7ff5080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff521e76640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff508004d40 0x7ff5080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff521e76640 1 -- 172.21.15.79:0/409886486 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff508003520 con 0x7ff5080039e0 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.467+0000 7ff522677640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff508004d40 0x7ff5080051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.468+0000 7ff521e76640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff508003e00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff518002230 tx=0x7ff508005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4023bca953e04c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.468+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51800a4a0 con 0x7ff5080039e0 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.468+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51802f070 con 0x7ff5080039e0 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.468+0000 7ff520e74640 1 -- 172.21.15.79:0/409886486 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51802bcb0 con 0x7ff5080039e0 2024-04-26T19:51:44.472 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.468+0000 7ff5240da640 1 -- 172.21.15.79:0/409886486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 msgr2=0x7ff508003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.468+0000 7ff5240da640 1 --2- 172.21.15.79:0/409886486 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff508003e00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff518002230 tx=0x7ff508005830 comp rx=0 tx=0).stop 2024-04-26T19:51:44.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.468+0000 7ff522677640 1 -- 172.21.15.79:0/409886486 reap_dead start 2024-04-26T19:51:44.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.468+0000 7ff5240da640 1 -- 172.21.15.79:0/409886486 shutdown_connections 2024-04-26T19:51:44.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.468+0000 7ff5240da640 1 -- 172.21.15.79:0/409886486 >> 172.21.15.79:0/409886486 conn(0x7ff51c0b39d0 msgr2=0x7ff51c06feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:44.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 -- 172.21.15.79:0/409886486 shutdown_connections 2024-04-26T19:51:44.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 -- 172.21.15.79:0/409886486 wait complete. 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 Processor -- start 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 -- start start 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff51c15efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c159840 0x7ff51c157fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c14ef20 0x7ff51c15cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c156bd0 con 0x7ff5080039e0 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7ff51c156a50 con 0x7ff51c159840 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff5240da640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7ff51c0cfd60 con 0x7ff51c14ef20 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff521e76640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff51c15efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff521e76640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff51c15efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54088/0 (socket says 172.21.15.79:54088) 2024-04-26T19:51:44.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.470+0000 7ff521e76640 1 -- 172.21.15.79:0/3440694760 learned_addr learned my addr 172.21.15.79:0/3440694760 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:44.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff522677640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c14ef20 0x7ff51c15cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff521675640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c159840 0x7ff51c157fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff521e76640 1 -- 172.21.15.79:0/3440694760 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c14ef20 msgr2=0x7ff51c15cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff521e76640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c14ef20 0x7ff51c15cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff521e76640 1 -- 172.21.15.79:0/3440694760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c159840 msgr2=0x7ff51c157fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff521e76640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c159840 0x7ff51c157fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff521e76640 1 -- 172.21.15.79:0/3440694760 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51c158500 con 0x7ff5080039e0 2024-04-26T19:51:44.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff522677640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c14ef20 0x7ff51c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:44.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff521e76640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff51c15efe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff51800a940 tx=0x7ff518001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:44.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff5127fc640 1 -- 172.21.15.79:0/3440694760 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518034670 con 0x7ff5080039e0 2024-04-26T19:51:44.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff5127fc640 1 -- 172.21.15.79:0/3440694760 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff518034810 con 0x7ff5080039e0 2024-04-26T19:51:44.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51c1587a0 con 0x7ff5080039e0 2024-04-26T19:51:44.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff5127fc640 1 -- 172.21.15.79:0/3440694760 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff518034bc0 con 0x7ff5080039e0 2024-04-26T19:51:44.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.472+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff51c158d60 con 0x7ff5080039e0 2024-04-26T19:51:44.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.473+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51c159c20 con 0x7ff5080039e0 2024-04-26T19:51:44.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.473+0000 7ff5127fc640 1 -- 172.21.15.79:0/3440694760 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff51803f440 con 0x7ff5080039e0 2024-04-26T19:51:44.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.473+0000 7ff5127fc640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff50006c920 0x7ff50006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.473+0000 7ff521675640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff50006c920 0x7ff50006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.473+0000 7ff5127fc640 1 -- 172.21.15.79:0/3440694760 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff5180ac780 con 0x7ff5080039e0 2024-04-26T19:51:44.479 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.473+0000 7ff521675640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff50006c920 0x7ff50006ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff51c155290 tx=0x7ff50c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:44.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.477+0000 7ff5127fc640 1 -- 172.21.15.79:0/3440694760 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7ff51807aab0 con 0x7ff5080039e0 2024-04-26T19:51:44.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.655+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff51c160020 con 0x7ff5080039e0 2024-04-26T19:51:44.656 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.657+0000 7ff5127fc640 1 -- 172.21.15.79:0/3440694760 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16907 (secure 0 0 0) 0x7ff51807a3b0 con 0x7ff5080039e0 2024-04-26T19:51:44.656 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:44.657 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":38,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:43.380056+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:51:41.583741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T19:51:42.339687+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:44.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff50006c920 msgr2=0x7ff50006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff50006c920 0x7ff50006ede0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff51c155290 tx=0x7ff50c008040 comp rx=0 tx=0).stop 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 msgr2=0x7ff51c15efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff51c15efe0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff51800a940 tx=0x7ff518001dc0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 shutdown_connections 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7ff50006c920 0x7ff50006ede0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7ff51c14ef20 0x7ff51c15cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7ff51c159840 0x7ff51c157fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 --2- 172.21.15.79:0/3440694760 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7ff5080039e0 0x7ff51c15efe0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 >> 172.21.15.79:0/3440694760 conn(0x7ff51c0b39d0 msgr2=0x7ff51c0b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 shutdown_connections 2024-04-26T19:51:44.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.659+0000 7ff5240da640 1 -- 172.21.15.79:0/3440694760 wait complete. 2024-04-26T19:51:44.669 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-26T19:51:44.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.769+0000 7f03e4540640 1 Processor -- start 2024-04-26T19:51:44.769 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.769+0000 7f03e4540640 1 -- start start 2024-04-26T19:51:44.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e4540640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc148f30 0x7f03dc0b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e4540640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f03dc06c350 0x7f03dc06c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e4540640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03dc0b7e60 con 0x7f03dc0b7620 2024-04-26T19:51:44.770 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e4540640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f03dc06ccf0 con 0x7f03dc06c350 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e4540640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0b8140 con 0x7f03dc148f30 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e1adb640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f03dc0b7620 0x7f03dc0b8f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57244/0 (socket says 172.21.15.79:57244) 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e1adb640 1 -- 172.21.15.79:0/1548801648 learned_addr learned my addr 172.21.15.79:0/1548801648 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e22dc640 1 --2- 172.21.15.79:0/1548801648 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc148f30 0x7f03dc0b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e2add640 1 --2- 172.21.15.79:0/1548801648 >> v2:172.21.15.83:3300/0 conn(0x7f03dc06c350 0x7f03dc06c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e12da640 1 -- 172.21.15.79:0/1548801648 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1927279451 0 0) 0x7f03dc0b7e60 con 0x7f03dc0b7620 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.770+0000 7f03e12da640 1 -- 172.21.15.79:0/1548801648 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03c8003520 con 0x7f03dc0b7620 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.771+0000 7f03e22dc640 1 -- 172.21.15.79:0/1548801648 >> v2:172.21.15.83:3300/0 conn(0x7f03dc06c350 msgr2=0x7f03dc06c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.771+0000 7f03e22dc640 1 --2- 172.21.15.79:0/1548801648 >> v2:172.21.15.83:3300/0 conn(0x7f03dc06c350 0x7f03dc06c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.771+0000 7f03e12da640 1 -- 172.21.15.79:0/1548801648 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3213762838 0 0) 0x7f03c8003520 con 0x7f03dc0b7620 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.771+0000 7f03e22dc640 1 -- 172.21.15.79:0/1548801648 >> v1:172.21.15.79:6789/0 conn(0x7f03dc0b7620 legacy=0x7f03dc0b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.771+0000 7f03e22dc640 1 -- 172.21.15.79:0/1548801648 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc0b8420 con 0x7f03dc148f30 2024-04-26T19:51:44.771 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.771+0000 7f03e2add640 1 --2- 172.21.15.79:0/1548801648 >> v2:172.21.15.83:3300/0 conn(0x7f03dc06c350 0x7f03dc06c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.771+0000 7f03e22dc640 1 --2- 172.21.15.79:0/1548801648 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc148f30 0x7f03dc0b6630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009d90 tx=0x7f03d802a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a5a5924d9be1d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e12da640 1 -- 172.21.15.79:0/1548801648 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03cc001670 con 0x7f03dc148f30 2024-04-26T19:51:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e12da640 1 -- 172.21.15.79:0/1548801648 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d8007540 con 0x7f03dc148f30 2024-04-26T19:51:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e12da640 1 -- 172.21.15.79:0/1548801648 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d80078f0 con 0x7f03dc148f30 2024-04-26T19:51:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e4540640 1 -- 172.21.15.79:0/1548801648 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc148f30 msgr2=0x7f03dc0b6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e4540640 1 --2- 172.21.15.79:0/1548801648 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc148f30 0x7f03dc0b6630 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f03d8009d90 tx=0x7f03d802a5f0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e4540640 1 -- 172.21.15.79:0/1548801648 shutdown_connections 2024-04-26T19:51:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e4540640 1 --2- 172.21.15.79:0/1548801648 >> v2:172.21.15.83:3300/0 conn(0x7f03dc06c350 0x7f03dc06c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.773 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e4540640 1 --2- 172.21.15.79:0/1548801648 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc148f30 0x7f03dc0b6630 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e4540640 1 -- 172.21.15.79:0/1548801648 >> 172.21.15.79:0/1548801648 conn(0x7f03dc073b40 msgr2=0x7f03dc073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:44.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e4540640 1 -- 172.21.15.79:0/1548801648 shutdown_connections 2024-04-26T19:51:44.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.772+0000 7f03e4540640 1 -- 172.21.15.79:0/1548801648 wait complete. 2024-04-26T19:51:44.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.773+0000 7f03e4540640 1 Processor -- start 2024-04-26T19:51:44.774 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.773+0000 7f03e4540640 1 -- start start 2024-04-26T19:51:44.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e4540640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc06c350 0x7f03dc1f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e4540640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc0b7620 0x7f03dc1414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e4540640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc148f30 0x7f03dc141a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.776 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e4540640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0b7a60 con 0x7f03dc148f30 2024-04-26T19:51:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e4540640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc0b89e0 con 0x7f03dc06c350 2024-04-26T19:51:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e4540640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f03dc0b7be0 con 0x7f03dc0b7620 2024-04-26T19:51:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e2add640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc148f30 0x7f03dc141a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e2add640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc148f30 0x7f03dc141a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54102/0 (socket says 172.21.15.79:54102) 2024-04-26T19:51:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e2add640 1 -- 172.21.15.79:0/2022027439 learned_addr learned my addr 172.21.15.79:0/2022027439 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e22dc640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc06c350 0x7f03dc1f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e1adb640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc0b7620 0x7f03dc1414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e2add640 1 -- 172.21.15.79:0/2022027439 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc0b7620 msgr2=0x7f03dc1414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.777 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e2add640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc0b7620 0x7f03dc1414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e2add640 1 -- 172.21.15.79:0/2022027439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc06c350 msgr2=0x7f03dc1f4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e2add640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc06c350 0x7f03dc1f4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.774+0000 7f03e2add640 1 -- 172.21.15.79:0/2022027439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc13ee50 con 0x7f03dc148f30 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.775+0000 7f03e2add640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc148f30 0x7f03dc141a10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f03d400e930 tx=0x7f03d400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.775+0000 7f03d2ffd640 1 -- 172.21.15.79:0/2022027439 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d4017070 con 0x7f03dc148f30 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.775+0000 7f03d2ffd640 1 -- 172.21.15.79:0/2022027439 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d400ca00 con 0x7f03dc148f30 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.775+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc13f130 con 0x7f03dc148f30 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.775+0000 7f03d2ffd640 1 -- 172.21.15.79:0/2022027439 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d400cdd0 con 0x7f03dc148f30 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.775+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc13f6f0 con 0x7f03dc148f30 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.776+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03dc0506c0 con 0x7f03dc148f30 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.776+0000 7f03d2ffd640 1 -- 172.21.15.79:0/2022027439 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03d4013470 con 0x7f03dc148f30 2024-04-26T19:51:44.778 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.777+0000 7f03d2ffd640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f03c006c8f0 0x7f03c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:44.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.777+0000 7f03d2ffd640 1 -- 172.21.15.79:0/2022027439 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f03d408bb00 con 0x7f03dc148f30 2024-04-26T19:51:44.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.777+0000 7f03e22dc640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f03c006c8f0 0x7f03c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:44.779 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.777+0000 7f03e22dc640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f03c006c8f0 0x7f03c006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f03d8034000 tx=0x7f03d802f000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:44.780 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.781+0000 7f03d2ffd640 1 -- 172.21.15.79:0/2022027439 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f03d40598a0 con 0x7f03dc148f30 2024-04-26T19:51:44.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:44.993+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f03dc148530 con 0x7f03dc148f30 2024-04-26T19:51:45.038 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.038+0000 7f03d2ffd640 1 -- 172.21.15.79:0/2022027439 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f03d40600c0 con 0x7f03dc148f30 2024-04-26T19:51:45.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f03c006c8f0 msgr2=0x7f03c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:45.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f03c006c8f0 0x7f03c006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f03d8034000 tx=0x7f03d802f000 comp rx=0 tx=0).stop 2024-04-26T19:51:45.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc148f30 msgr2=0x7f03dc141a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:45.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc148f30 0x7f03dc141a10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f03d400e930 tx=0x7f03d400edf0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 shutdown_connections 2024-04-26T19:51:45.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f03c006c8f0 0x7f03c006edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f03dc148f30 0x7f03dc141a10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f03dc0b7620 0x7f03dc1414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 --2- 172.21.15.79:0/2022027439 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f03dc06c350 0x7f03dc1f4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 >> 172.21.15.79:0/2022027439 conn(0x7f03dc073b40 msgr2=0x7f03dc146470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:45.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 shutdown_connections 2024-04-26T19:51:45.042 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.041+0000 7f03e4540640 1 -- 172.21.15.79:0/2022027439 wait complete. 2024-04-26T19:51:45.052 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-26T19:51:45.052 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-26T19:51:45.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.151+0000 7f06ba351640 1 Processor -- start 2024-04-26T19:51:45.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.151+0000 7f06ba351640 1 -- start start 2024-04-26T19:51:45.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.151+0000 7f06ba351640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 0x7f06b4163090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:45.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.151+0000 7f06ba351640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f06b406f6b0 0x7f06b406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:45.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.151+0000 7f06ba351640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06b4163ff0 con 0x7f06b4160590 2024-04-26T19:51:45.152 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.151+0000 7f06ba351640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f06b40ac4d0 con 0x7f06b406f6b0 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.151+0000 7f06ba351640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f06b4160a10 con 0x7f06b4159c10 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.152+0000 7f06b88ee640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f06b406f6b0 0x7f06b406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.152+0000 7f06b88ee640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f06b406f6b0 0x7f06b406fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54112/0 (socket says 172.21.15.79:54112) 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.152+0000 7f06b88ee640 1 -- 172.21.15.79:0/30749246 learned_addr learned my addr 172.21.15.79:0/30749246 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.152+0000 7f06b3fff640 1 --2- 172.21.15.79:0/30749246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 0x7f06b4163090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.152+0000 7f06b88ee640 1 -- 172.21.15.79:0/30749246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 msgr2=0x7f06b4163090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.152+0000 7f06b88ee640 1 --2- 172.21.15.79:0/30749246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 0x7f06b4163090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.152+0000 7f06b88ee640 1 -- 172.21.15.79:0/30749246 >> v1:172.21.15.83:6789/0 conn(0x7f06b4160590 legacy=0x7f06b40ab0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.152+0000 7f06b88ee640 1 -- 172.21.15.79:0/30749246 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b4163750 con 0x7f06b406f6b0 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06b88ee640 1 --2- 172.21.15.79:0/30749246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 0x7f06b406fb40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009e50 tx=0x7f06ac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8d72fce2bb758b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06b2ffd640 1 -- 172.21.15.79:0/30749246 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac036070 con 0x7f06b406f6b0 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06b2ffd640 1 -- 172.21.15.79:0/30749246 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06ac031600 con 0x7f06b406f6b0 2024-04-26T19:51:45.153 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06b2ffd640 1 -- 172.21.15.79:0/30749246 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0319d0 con 0x7f06b406f6b0 2024-04-26T19:51:45.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06ba351640 1 -- 172.21.15.79:0/30749246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 msgr2=0x7f06b406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:45.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06ba351640 1 --2- 172.21.15.79:0/30749246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 0x7f06b406fb40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009e50 tx=0x7f06ac02a680 comp rx=0 tx=0).stop 2024-04-26T19:51:45.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06ba351640 1 -- 172.21.15.79:0/30749246 shutdown_connections 2024-04-26T19:51:45.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06ba351640 1 --2- 172.21.15.79:0/30749246 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 0x7f06b406fb40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.156 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06ba351640 1 --2- 172.21.15.79:0/30749246 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 0x7f06b4163090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06ba351640 1 -- 172.21.15.79:0/30749246 >> 172.21.15.79:0/30749246 conn(0x7f06b40d24f0 msgr2=0x7f06b40d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06ba351640 1 -- 172.21.15.79:0/30749246 shutdown_connections 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.153+0000 7f06ba351640 1 -- 172.21.15.79:0/30749246 wait complete. 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.155+0000 7f06ba351640 1 Processor -- start 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.155+0000 7f06ba351640 1 -- start start 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.155+0000 7f06ba351640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 0x7f06b4154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.155+0000 7f06ba351640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 0x7f06b4147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.155+0000 7f06ba351640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06b4160590 0x7f06b4159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.155+0000 7f06ba351640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f06b40ab800 con 0x7f06b406f6b0 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.155+0000 7f06ba351640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06b4163d10 con 0x7f06b4160590 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.155+0000 7f06ba351640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f06b40ab980 con 0x7f06b4159c10 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.155+0000 7f06b3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 0x7f06b4154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b3fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 0x7f06b4154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54126/0 (socket says 172.21.15.79:54126) 2024-04-26T19:51:45.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b3fff640 1 -- 172.21.15.79:0/3001006191 learned_addr learned my addr 172.21.15.79:0/3001006191 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b88ee640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06b4160590 0x7f06b4159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b37fe640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 0x7f06b4147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b3fff640 1 -- 172.21.15.79:0/3001006191 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 msgr2=0x7f06b4147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b3fff640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 0x7f06b4147cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b3fff640 1 -- 172.21.15.79:0/3001006191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06b4160590 msgr2=0x7f06b4159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b3fff640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06b4160590 0x7f06b4159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b3fff640 1 -- 172.21.15.79:0/3001006191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b415a790 con 0x7f06b406f6b0 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b3fff640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 0x7f06b4154e00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f06a800e9b0 tx=0x7f06a800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b0ff9640 1 -- 172.21.15.79:0/3001006191 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a8017070 con 0x7f06b406f6b0 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b0ff9640 1 -- 172.21.15.79:0/3001006191 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06a800c9a0 con 0x7f06b406f6b0 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b4155f70 con 0x7f06b406f6b0 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06b0ff9640 1 -- 172.21.15.79:0/3001006191 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06a800cd90 con 0x7f06b406f6b0 2024-04-26T19:51:45.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.156+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b41564f0 con 0x7f06b406f6b0 2024-04-26T19:51:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.157+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b40506c0 con 0x7f06b406f6b0 2024-04-26T19:51:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.158+0000 7f06b0ff9640 1 -- 172.21.15.79:0/3001006191 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06a801a070 con 0x7f06b406f6b0 2024-04-26T19:51:45.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.158+0000 7f06b0ff9640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f06a006c8f0 0x7f06a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:45.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.158+0000 7f06b0ff9640 1 -- 172.21.15.79:0/3001006191 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f06a808bba0 con 0x7f06b406f6b0 2024-04-26T19:51:45.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.158+0000 7f06b37fe640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f06a006c8f0 0x7f06a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:45.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.158+0000 7f06b37fe640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f06a006c8f0 0x7f06a006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06b40d10b0 tx=0x7f06a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:45.162 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.162+0000 7f06b0ff9640 1 -- 172.21.15.79:0/3001006191 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f06a8059880 con 0x7f06b406f6b0 2024-04-26T19:51:45.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:45.374+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f06b40b1130 con 0x7f06b406f6b0 2024-04-26T19:51:46.052 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.052+0000 7f06b0ff9640 1 -- 172.21.15.79:0/3001006191 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f06a805db80 con 0x7f06b406f6b0 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f06a006c8f0 msgr2=0x7f06a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f06a006c8f0 0x7f06a006edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f06b40d10b0 tx=0x7f06a4009040 comp rx=0 tx=0).stop 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 msgr2=0x7f06b4154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 0x7f06b4154e00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f06a800e9b0 tx=0x7f06a800ee70 comp rx=0 tx=0).stop 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 shutdown_connections 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f06a006c8f0 0x7f06a006edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f06b4160590 0x7f06b4159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06b4159c10 0x7f06b4147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 --2- 172.21.15.79:0/3001006191 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f06b406f6b0 0x7f06b4154e00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 >> 172.21.15.79:0/3001006191 conn(0x7f06b40d24f0 msgr2=0x7f06b40cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:46.055 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 shutdown_connections 2024-04-26T19:51:46.056 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.054+0000 7f06ba351640 1 -- 172.21.15.79:0/3001006191 wait complete. 2024-04-26T19:51:46.066 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-26T19:51:46.069 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:51:46.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.166+0000 7efd0b7c1640 1 Processor -- start 2024-04-26T19:51:46.167 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.166+0000 7efd0b7c1640 1 -- start start 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.167+0000 7efd0b7c1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 0x7efd040d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.167+0000 7efd0b7c1640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efd0406ff00 0x7efd040d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.167+0000 7efd0b7c1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7efd041488c0 0x7efd04148d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.167+0000 7efd0b7c1640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7efd040d3e60 con 0x7efd0406ff00 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.167+0000 7efd0b7c1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7efd040b1290 con 0x7efd041488c0 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.167+0000 7efd0b7c1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7efd040d4090 con 0x7efd0406fb20 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.167+0000 7efd08d5c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efd0406ff00 0x7efd040d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.167+0000 7efd08d5c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7efd0406ff00 0x7efd040d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54128/0 (socket says 172.21.15.79:54128) 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.167+0000 7efd08d5c640 1 -- 172.21.15.79:0/2037910184 learned_addr learned my addr 172.21.15.79:0/2037910184 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd0955d640 1 --2- 172.21.15.79:0/2037910184 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 0x7efd040d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd09d5e640 1 --2- 172.21.15.79:0/2037910184 >> v2:172.21.15.83:3300/0 conn(0x7efd041488c0 0x7efd04148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd08d5c640 1 -- 172.21.15.79:0/2037910184 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 msgr2=0x7efd040d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd08d5c640 1 --2- 172.21.15.79:0/2037910184 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 0x7efd040d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd08d5c640 1 -- 172.21.15.79:0/2037910184 >> v2:172.21.15.83:3300/0 conn(0x7efd041488c0 msgr2=0x7efd04148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd08d5c640 1 --2- 172.21.15.79:0/2037910184 >> v2:172.21.15.83:3300/0 conn(0x7efd041488c0 0x7efd04148d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd08d5c640 1 -- 172.21.15.79:0/2037910184 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd040d4370 con 0x7efd0406ff00 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd0955d640 1 --2- 172.21.15.79:0/2037910184 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 0x7efd040d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd09d5e640 1 --2- 172.21.15.79:0/2037910184 >> v2:172.21.15.83:3300/0 conn(0x7efd041488c0 0x7efd04148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efd08d5c640 1 --2- 172.21.15.79:0/2037910184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd0406ff00 0x7efd040d3920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efd040b11e0 tx=0x7efcf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26c27a86b927b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.168+0000 7efcfbfff640 1 -- 172.21.15.79:0/2037910184 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4037070 con 0x7efd0406ff00 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efcfbfff640 1 -- 172.21.15.79:0/2037910184 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf4032440 con 0x7efd0406ff00 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efcfbfff640 1 -- 172.21.15.79:0/2037910184 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4032810 con 0x7efd0406ff00 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efd0b7c1640 1 -- 172.21.15.79:0/2037910184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd0406ff00 msgr2=0x7efd040d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/2037910184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd0406ff00 0x7efd040d3920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7efd040b11e0 tx=0x7efcf402a270 comp rx=0 tx=0).stop 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efd0b7c1640 1 -- 172.21.15.79:0/2037910184 shutdown_connections 2024-04-26T19:51:46.171 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/2037910184 >> v2:172.21.15.83:3300/0 conn(0x7efd041488c0 0x7efd04148d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/2037910184 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd0406ff00 0x7efd040d3920 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/2037910184 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 0x7efd040d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efd0b7c1640 1 -- 172.21.15.79:0/2037910184 >> 172.21.15.79:0/2037910184 conn(0x7efd04071c30 msgr2=0x7efd0406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:46.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efd0b7c1640 1 -- 172.21.15.79:0/2037910184 shutdown_connections 2024-04-26T19:51:46.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.169+0000 7efd0b7c1640 1 -- 172.21.15.79:0/2037910184 wait complete. 2024-04-26T19:51:46.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.170+0000 7efd0b7c1640 1 Processor -- start 2024-04-26T19:51:46.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.170+0000 7efd0b7c1640 1 -- start start 2024-04-26T19:51:46.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd0b7c1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 0x7efd041f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd0b7c1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0406ff00 0x7efd0415ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.175 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd0b7c1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd041488c0 0x7efd0413f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd0b7c1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7efd040d4c10 con 0x7efd041488c0 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd0b7c1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7efd040d4930 con 0x7efd0406ff00 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd0b7c1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7efd04149270 con 0x7efd0406fb20 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd09d5e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd041488c0 0x7efd0413f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd09d5e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd041488c0 0x7efd0413f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54142/0 (socket says 172.21.15.79:54142) 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd09d5e640 1 -- 172.21.15.79:0/1787343986 learned_addr learned my addr 172.21.15.79:0/1787343986 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd08d5c640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0406ff00 0x7efd0415ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.171+0000 7efd0955d640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 0x7efd041f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efd09d5e640 1 -- 172.21.15.79:0/1787343986 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 msgr2=0x7efd041f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efd09d5e640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 0x7efd041f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efd09d5e640 1 -- 172.21.15.79:0/1787343986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0406ff00 msgr2=0x7efd0415ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efd09d5e640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0406ff00 0x7efd0415ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efd09d5e640 1 -- 172.21.15.79:0/1787343986 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0413c590 con 0x7efd041488c0 2024-04-26T19:51:46.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efd09d5e640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd041488c0 0x7efd0413f3c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00c7f0 tx=0x7efcfc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efcf9ffb640 1 -- 172.21.15.79:0/1787343986 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc015070 con 0x7efd041488c0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efcf9ffb640 1 -- 172.21.15.79:0/1787343986 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcfc007820 con 0x7efd041488c0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efcf9ffb640 1 -- 172.21.15.79:0/1787343986 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc007c50 con 0x7efd041488c0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0413c860 con 0x7efd041488c0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.172+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd041417d0 con 0x7efd041488c0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.173+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd040506c0 con 0x7efd041488c0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.173+0000 7efcf9ffb640 1 -- 172.21.15.79:0/1787343986 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efcfc018070 con 0x7efd041488c0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.174+0000 7efcf9ffb640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efce806c8f0 0x7efce806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.174+0000 7efd0955d640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efce806c8f0 0x7efce806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.174+0000 7efcf9ffb640 1 -- 172.21.15.79:0/1787343986 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7efcfc08bda0 con 0x7efd041488c0 2024-04-26T19:51:46.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.174+0000 7efd0955d640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efce806c8f0 0x7efce806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efd0000f500 tx=0x7efd00007700 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:46.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.178+0000 7efcf9ffb640 1 -- 172.21.15.79:0/1787343986 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7efd0413c860 con 0x7efd041488c0 2024-04-26T19:51:46.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.390+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd040d3700 con 0x7efd041488c0 2024-04-26T19:51:46.391 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.391+0000 7efcf9ffb640 1 -- 172.21.15.79:0/1787343986 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7efcfc0599f0 con 0x7efd041488c0 2024-04-26T19:51:46.391 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:46.392 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2061494256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":2061494256},{"type":"v1","addr":"172.21.15.83:6835","nonce":2061494256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/4034640420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":4034640420},{"type":"v1","addr":"172.21.15.83:6834","nonce":4034640420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/238749789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":238749789},{"type":"v1","addr":"172.21.15.79:6836","nonce":238749789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:51:43.380019+0000","modified":"2024-04-26T19:51:45.374734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/2334214524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":2334214524},{"type":"v1","addr":"172.21.15.79:6837","nonce":2334214524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5343,"qdb_cluster":[5343]},"id":2}]} 2024-04-26T19:51:46.392 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2024-04-26T19:51:46.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efce806c8f0 msgr2=0x7efce806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efce806c8f0 0x7efce806edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efd0000f500 tx=0x7efd00007700 comp rx=0 tx=0).stop 2024-04-26T19:51:46.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd041488c0 msgr2=0x7efd0413f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd041488c0 0x7efd0413f3c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00c7f0 tx=0x7efcfc00ccb0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 shutdown_connections 2024-04-26T19:51:46.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7efce806c8f0 0x7efce806edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7efd041488c0 0x7efd0413f3c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7efd0406ff00 0x7efd0415ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 --2- 172.21.15.79:0/1787343986 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7efd0406fb20 0x7efd041f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 >> 172.21.15.79:0/1787343986 conn(0x7efd04071c30 msgr2=0x7efd04160bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:46.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 shutdown_connections 2024-04-26T19:51:46.395 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.393+0000 7efd0b7c1640 1 -- 172.21.15.79:0/1787343986 wait complete. 2024-04-26T19:51:46.404 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:51:46.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.500+0000 7f5159797640 1 Processor -- start 2024-04-26T19:51:46.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.500+0000 7f5159797640 1 -- start start 2024-04-26T19:51:46.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.501+0000 7f5159797640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f515415d120 0x7f51540d0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.501+0000 7f5159797640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5154070050 0x7f51540706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.501+0000 7f5159797640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51540704b0 con 0x7f51540d23c0 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.501+0000 7f5159797640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f5154070f60 con 0x7f515415d120 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.501+0000 7f5159797640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f515406fbc0 con 0x7f5154070050 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.501+0000 7f5152ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f515415d120 0x7f51540d0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.501+0000 7f5152ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f515415d120 0x7f51540d0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54146/0 (socket says 172.21.15.79:54146) 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.501+0000 7f5152ffd640 1 -- 172.21.15.79:0/79397705 learned_addr learned my addr 172.21.15.79:0/79397705 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.502+0000 7f51527fc640 1 --2- 172.21.15.79:0/79397705 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5154070050 0x7f51540706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.502+0000 7f5152ffd640 1 -- 172.21.15.79:0/79397705 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5154070050 msgr2=0x7f51540706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.502+0000 7f5152ffd640 1 --2- 172.21.15.79:0/79397705 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5154070050 0x7f51540706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.502+0000 7f5152ffd640 1 -- 172.21.15.79:0/79397705 >> v1:172.21.15.83:6789/0 conn(0x7f51540d23c0 legacy=0x7f51540d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.502+0000 7f5152ffd640 1 -- 172.21.15.79:0/79397705 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515406f290 con 0x7f515415d120 2024-04-26T19:51:46.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.502+0000 7f5152ffd640 1 --2- 172.21.15.79:0/79397705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 0x7f51540d0bc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5144009e50 tx=0x7f514402a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce66cedc45aacb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:46.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5151ffb640 1 -- 172.21.15.79:0/79397705 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5144036070 con 0x7f515415d120 2024-04-26T19:51:46.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5151ffb640 1 -- 172.21.15.79:0/79397705 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5144031600 con 0x7f515415d120 2024-04-26T19:51:46.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5151ffb640 1 -- 172.21.15.79:0/79397705 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440319d0 con 0x7f515415d120 2024-04-26T19:51:46.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5159797640 1 -- 172.21.15.79:0/79397705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 msgr2=0x7f51540d0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5159797640 1 --2- 172.21.15.79:0/79397705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 0x7f51540d0bc0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f5144009e50 tx=0x7f514402a5b0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5159797640 1 -- 172.21.15.79:0/79397705 shutdown_connections 2024-04-26T19:51:46.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5159797640 1 --2- 172.21.15.79:0/79397705 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5154070050 0x7f51540706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5159797640 1 --2- 172.21.15.79:0/79397705 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 0x7f51540d0bc0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5159797640 1 -- 172.21.15.79:0/79397705 >> 172.21.15.79:0/79397705 conn(0x7f5154073b20 msgr2=0x7f5154073f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:46.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5159797640 1 -- 172.21.15.79:0/79397705 shutdown_connections 2024-04-26T19:51:46.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.503+0000 7f5159797640 1 -- 172.21.15.79:0/79397705 wait complete. 2024-04-26T19:51:46.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.504+0000 7f5159797640 1 Processor -- start 2024-04-26T19:51:46.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.504+0000 7f5159797640 1 -- start start 2024-04-26T19:51:46.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f5159797640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5154070050 0x7f515414d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f5159797640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f51540d23c0 0x7f51541517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f5159797640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 0x7f5154153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f5159797640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5154070be0 con 0x7f515415d120 2024-04-26T19:51:46.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f5159797640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f515406f850 con 0x7f5154070050 2024-04-26T19:51:46.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f5159797640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f515415d500 con 0x7f51540d23c0 2024-04-26T19:51:46.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f51537fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 0x7f5154153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f51537fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 0x7f5154153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54162/0 (socket says 172.21.15.79:54162) 2024-04-26T19:51:46.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f51537fe640 1 -- 172.21.15.79:0/1481859485 learned_addr learned my addr 172.21.15.79:0/1481859485 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:46.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f51527fc640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f51540d23c0 0x7f51541517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f5152ffd640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5154070050 0x7f515414d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.505+0000 7f51537fe640 1 -- 172.21.15.79:0/1481859485 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f51540d23c0 msgr2=0x7f51541517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f51537fe640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f51540d23c0 0x7f51541517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f51537fe640 1 -- 172.21.15.79:0/1481859485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5154070050 msgr2=0x7f515414d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f51537fe640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5154070050 0x7f515414d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f51537fe640 1 -- 172.21.15.79:0/1481859485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f515414dce0 con 0x7f515415d120 2024-04-26T19:51:46.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f51537fe640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 0x7f5154153d70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c410 tx=0x7f514c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:46.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f513b7fe640 1 -- 172.21.15.79:0/1481859485 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c011790 con 0x7f515415d120 2024-04-26T19:51:46.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f513b7fe640 1 -- 172.21.15.79:0/1481859485 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f514c017070 con 0x7f515415d120 2024-04-26T19:51:46.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5154152170 con 0x7f515415d120 2024-04-26T19:51:46.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f513b7fe640 1 -- 172.21.15.79:0/1481859485 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c011bb0 con 0x7f515415d120 2024-04-26T19:51:46.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.506+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5154152730 con 0x7f515415d120 2024-04-26T19:51:46.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.507+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51540506c0 con 0x7f515415d120 2024-04-26T19:51:46.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.507+0000 7f513b7fe640 1 -- 172.21.15.79:0/1481859485 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f514c01c460 con 0x7f515415d120 2024-04-26T19:51:46.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.508+0000 7f513b7fe640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f514006c8f0 0x7f514006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.508+0000 7f513b7fe640 1 -- 172.21.15.79:0/1481859485 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f514c08b930 con 0x7f515415d120 2024-04-26T19:51:46.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.508+0000 7f5152ffd640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f514006c8f0 0x7f514006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.508+0000 7f5152ffd640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f514006c8f0 0x7f514006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5144033000 tx=0x7f5144009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:46.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.512+0000 7f513b7fe640 1 -- 172.21.15.79:0/1481859485 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f514c021020 con 0x7f515415d120 2024-04-26T19:51:46.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.685+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f515406ed90 con 0x7f515415d120 2024-04-26T19:51:46.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.686+0000 7f513b7fe640 1 -- 172.21.15.79:0/1481859485 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16894 (secure 0 0 0) 0x7f514c059bc0 con 0x7f515415d120 2024-04-26T19:51:46.687 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:46.687 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":39,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:45.037918+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:51:41.583741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T19:51:42.339687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:51:46.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f514006c8f0 msgr2=0x7f514006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f514006c8f0 0x7f514006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5144033000 tx=0x7f5144009000 comp rx=0 tx=0).stop 2024-04-26T19:51:46.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 msgr2=0x7f5154153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 0x7f5154153d70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c410 tx=0x7f514c00c8d0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 shutdown_connections 2024-04-26T19:51:46.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f514006c8f0 0x7f514006edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f515415d120 0x7f5154153d70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f51540d23c0 0x7f51541517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 --2- 172.21.15.79:0/1481859485 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5154070050 0x7f515414d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 >> 172.21.15.79:0/1481859485 conn(0x7f5154073b20 msgr2=0x7f51541576c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:46.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 shutdown_connections 2024-04-26T19:51:46.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.688+0000 7f5159797640 1 -- 172.21.15.79:0/1481859485 wait complete. 2024-04-26T19:51:46.700 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-26T19:51:46.700 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:51:46.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.796+0000 7fda5a9f0640 1 Processor -- start 2024-04-26T19:51:46.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.796+0000 7fda5a9f0640 1 -- start start 2024-04-26T19:51:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.796+0000 7fda5a9f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda5406ff00 0x7fda54148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda5a9f0640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda540b1290 con 0x7fda5406fb20 2024-04-26T19:51:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda5a9f0640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda540d4090 con 0x7fda540d2cd0 2024-04-26T19:51:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda5a9f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fda540d3110 con 0x7fda5406ff00 2024-04-26T19:51:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda53fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fda5406fb20 0x7fda541488c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57248/0 (socket says 172.21.15.79:57248) 2024-04-26T19:51:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda53fff640 1 -- 172.21.15.79:0/2310880895 learned_addr learned my addr 172.21.15.79:0/2310880895 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda537fe640 1 --2- 172.21.15.79:0/2310880895 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda5406ff00 0x7fda54148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.798 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda52ffd640 1 -- 172.21.15.79:0/2310880895 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725387907 0 0) 0x7fda540b1290 con 0x7fda5406fb20 2024-04-26T19:51:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda52ffd640 1 -- 172.21.15.79:0/2310880895 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda3c003520 con 0x7fda5406fb20 2024-04-26T19:51:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda52ffd640 1 -- 172.21.15.79:0/2310880895 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 145514850 0 0) 0x7fda540d4090 con 0x7fda540d2cd0 2024-04-26T19:51:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda52ffd640 1 -- 172.21.15.79:0/2310880895 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda540b1290 con 0x7fda540d2cd0 2024-04-26T19:51:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda52ffd640 1 -- 172.21.15.79:0/2310880895 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1050596883 0 0) 0x7fda3c003520 con 0x7fda5406fb20 2024-04-26T19:51:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda537fe640 1 -- 172.21.15.79:0/2310880895 >> v1:172.21.15.83:6789/0 conn(0x7fda540d2cd0 legacy=0x7fda540aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda537fe640 1 -- 172.21.15.79:0/2310880895 >> v1:172.21.15.79:6789/0 conn(0x7fda5406fb20 legacy=0x7fda541488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.797+0000 7fda537fe640 1 -- 172.21.15.79:0/2310880895 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda540d4370 con 0x7fda5406ff00 2024-04-26T19:51:46.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.798+0000 7fda537fe640 1 --2- 172.21.15.79:0/2310880895 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda5406ff00 0x7fda54148fd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda480097d0 tx=0x7fda4802a230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d660aac36f5bb2c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:46.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.798+0000 7fda52ffd640 1 -- 172.21.15.79:0/2310880895 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda48037070 con 0x7fda5406ff00 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.798+0000 7fda52ffd640 1 -- 172.21.15.79:0/2310880895 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda4802acc0 con 0x7fda5406ff00 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.798+0000 7fda52ffd640 1 -- 172.21.15.79:0/2310880895 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44001670 con 0x7fda5406ff00 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.799+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2310880895 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda5406ff00 msgr2=0x7fda54148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.799+0000 7fda5a9f0640 1 --2- 172.21.15.79:0/2310880895 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda5406ff00 0x7fda54148fd0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fda480097d0 tx=0x7fda4802a230 comp rx=0 tx=0).stop 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.799+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2310880895 shutdown_connections 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.799+0000 7fda5a9f0640 1 --2- 172.21.15.79:0/2310880895 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda5406ff00 0x7fda54148fd0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.799+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2310880895 >> 172.21.15.79:0/2310880895 conn(0x7fda54071c30 msgr2=0x7fda5406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.799+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2310880895 shutdown_connections 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.799+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2310880895 wait complete. 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.800+0000 7fda5a9f0640 1 Processor -- start 2024-04-26T19:51:46.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.800+0000 7fda5a9f0640 1 -- start start 2024-04-26T19:51:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda5a9f0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda5406fb20 0x7fda54158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda5a9f0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda5406ff00 0x7fda54143bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda5a9f0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda540d2cd0 0x7fda54144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda5a9f0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fda540d4c10 con 0x7fda5406fb20 2024-04-26T19:51:46.804 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda5a9f0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fda540d4930 con 0x7fda5406ff00 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda5a9f0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fda540d3920 con 0x7fda540d2cd0 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda53fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda5406fb20 0x7fda54158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda53fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda5406fb20 0x7fda54158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54164/0 (socket says 172.21.15.79:54164) 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda53fff640 1 -- 172.21.15.79:0/2000562479 learned_addr learned my addr 172.21.15.79:0/2000562479 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda537fe640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda5406ff00 0x7fda54143bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda53fff640 1 -- 172.21.15.79:0/2000562479 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda540d2cd0 msgr2=0x7fda54144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda53fff640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda540d2cd0 0x7fda54144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda53fff640 1 -- 172.21.15.79:0/2000562479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda5406ff00 msgr2=0x7fda54143bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda53fff640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda5406ff00 0x7fda54143bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda53fff640 1 -- 172.21.15.79:0/2000562479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda541597d0 con 0x7fda5406fb20 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda53fff640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda5406fb20 0x7fda54158e60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fda440017e0 tx=0x7fda4400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:46.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda50ff9640 1 -- 172.21.15.79:0/2000562479 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44017070 con 0x7fda5406fb20 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda50ff9640 1 -- 172.21.15.79:0/2000562479 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda44005ad0 con 0x7fda5406fb20 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda54159ab0 con 0x7fda5406fb20 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda50ff9640 1 -- 172.21.15.79:0/2000562479 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda44007070 con 0x7fda5406fb20 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.801+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda5415a070 con 0x7fda5406fb20 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.802+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda540506c0 con 0x7fda5406fb20 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.803+0000 7fda50ff9640 1 -- 172.21.15.79:0/2000562479 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda4401a070 con 0x7fda5406fb20 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.803+0000 7fda50ff9640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda4006c8f0 0x7fda4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.803+0000 7fda50ff9640 1 -- 172.21.15.79:0/2000562479 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fda4408b0a0 con 0x7fda5406fb20 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.803+0000 7fda537fe640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda4006c8f0 0x7fda4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:46.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.804+0000 7fda537fe640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda4006c8f0 0x7fda4006edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fda48034000 tx=0x7fda4802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:46.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.807+0000 7fda50ff9640 1 -- 172.21.15.79:0/2000562479 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fda44058d80 con 0x7fda5406fb20 2024-04-26T19:51:46.977 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.977+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fda540d3700 con 0x7fda4006c8f0 2024-04-26T19:51:46.983 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.982+0000 7fda50ff9640 1 -- 172.21.15.79:0/2000562479 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240051 (secure 0 0 0) 0x7fda540d3700 con 0x7fda4006c8f0 2024-04-26T19:51:46.986 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:46.987 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda4006c8f0 msgr2=0x7fda4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda4006c8f0 0x7fda4006edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fda48034000 tx=0x7fda4802e000 comp rx=0 tx=0).stop 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda5406fb20 msgr2=0x7fda54158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda5406fb20 0x7fda54158e60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fda440017e0 tx=0x7fda4400cc00 comp rx=0 tx=0).stop 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 shutdown_connections 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fda4006c8f0 0x7fda4006edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fda540d2cd0 0x7fda54144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fda5406ff00 0x7fda54143bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 --2- 172.21.15.79:0/2000562479 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fda5406fb20 0x7fda54158e60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 >> 172.21.15.79:0/2000562479 conn(0x7fda54071c30 msgr2=0x7fda54141fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 shutdown_connections 2024-04-26T19:51:46.989 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:46.985+0000 7fda5a9f0640 1 -- 172.21.15.79:0/2000562479 wait complete. 2024-04-26T19:51:46.996 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-26T19:51:46.271542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220512,"kb_used_data":4332,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4435968,"data_stored":2562739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.246429"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.026344+0000","last_change":"2024-04-26T19:51:42.026344+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.026344+0000","last_undegraded":"2024-04-26T19:51:42.026344+0000","last_fullsized":"2024-04-26T19:51:42.026344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028843+0000","last_change":"2024-04-26T19:51:42.028843+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028843+0000","last_undegraded":"2024-04-26T19:51:42.028843+0000","last_fullsized":"2024-04-26T19:51:42.028843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031161+0000","last_change":"2024-04-26T19:51:42.031161+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031161+0000","last_undegraded":"2024-04-26T19:51:42.031161+0000","last_fullsized":"2024-04-26T19:51:42.031161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028023+0000","last_change":"2024-04-26T19:51:42.028023+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028023+0000","last_undegraded":"2024-04-26T19:51:42.028023+0000","last_fullsized":"2024-04-26T19:51:42.028023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030077+0000","last_change":"2024-04-26T19:51:42.030077+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030077+0000","last_undegraded":"2024-04-26T19:51:42.030077+0000","last_fullsized":"2024-04-26T19:51:42.030077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031054+0000","last_change":"2024-04-26T19:51:42.031054+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031054+0000","last_undegraded":"2024-04-26T19:51:42.031054+0000","last_fullsized":"2024-04-26T19:51:42.031054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031173+0000","last_change":"2024-04-26T19:51:42.031173+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031173+0000","last_undegraded":"2024-04-26T19:51:42.031173+0000","last_fullsized":"2024-04-26T19:51:42.031173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028030+0000","last_change":"2024-04-26T19:51:42.028030+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028030+0000","last_undegraded":"2024-04-26T19:51:42.028030+0000","last_fullsized":"2024-04-26T19:51:42.028030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033452+0000","last_change":"2024-04-26T19:51:42.033452+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033452+0000","last_undegraded":"2024-04-26T19:51:42.033452+0000","last_fullsized":"2024-04-26T19:51:42.033452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030122+0000","last_change":"2024-04-26T19:51:42.030122+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030122+0000","last_undegraded":"2024-04-26T19:51:42.030122+0000","last_fullsized":"2024-04-26T19:51:42.030122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030131+0000","last_change":"2024-04-26T19:51:42.030131+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030131+0000","last_undegraded":"2024-04-26T19:51:42.030131+0000","last_fullsized":"2024-04-26T19:51:42.030131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033525+0000","last_change":"2024-04-26T19:51:42.033525+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033525+0000","last_undegraded":"2024-04-26T19:51:42.033525+0000","last_fullsized":"2024-04-26T19:51:42.033525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033417+0000","last_change":"2024-04-26T19:51:42.033417+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033417+0000","last_undegraded":"2024-04-26T19:51:42.033417+0000","last_fullsized":"2024-04-26T19:51:42.033417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031159+0000","last_change":"2024-04-26T19:51:42.031159+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031159+0000","last_undegraded":"2024-04-26T19:51:42.031159+0000","last_fullsized":"2024-04-26T19:51:42.031159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.026344+0000","last_change":"2024-04-26T19:51:42.026344+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.026344+0000","last_undegraded":"2024-04-26T19:51:42.026344+0000","last_fullsized":"2024-04-26T19:51:42.026344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.032485+0000","last_change":"2024-04-26T19:51:42.032485+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.032485+0000","last_undegraded":"2024-04-26T19:51:42.032485+0000","last_fullsized":"2024-04-26T19:51:42.032485+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030545+0000","last_change":"2024-04-26T19:51:42.030545+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030545+0000","last_undegraded":"2024-04-26T19:51:42.030545+0000","last_fullsized":"2024-04-26T19:51:42.030545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028035+0000","last_change":"2024-04-26T19:51:42.028035+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028035+0000","last_undegraded":"2024-04-26T19:51:42.028035+0000","last_fullsized":"2024-04-26T19:51:42.028035+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028448+0000","last_change":"2024-04-26T19:51:42.028448+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028448+0000","last_undegraded":"2024-04-26T19:51:42.028448+0000","last_fullsized":"2024-04-26T19:51:42.028448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031161+0000","last_change":"2024-04-26T19:51:42.031161+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031161+0000","last_undegraded":"2024-04-26T19:51:42.031161+0000","last_fullsized":"2024-04-26T19:51:42.031161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028450+0000","last_change":"2024-04-26T19:51:42.028450+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028450+0000","last_undegraded":"2024-04-26T19:51:42.028450+0000","last_fullsized":"2024-04-26T19:51:42.028450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028357+0000","last_change":"2024-04-26T19:51:42.028357+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028357+0000","last_undegraded":"2024-04-26T19:51:42.028357+0000","last_fullsized":"2024-04-26T19:51:42.028357+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031981+0000","last_change":"2024-04-26T19:51:42.031981+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031981+0000","last_undegraded":"2024-04-26T19:51:42.031981+0000","last_fullsized":"2024-04-26T19:51:42.031981+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028368+0000","last_change":"2024-04-26T19:51:42.028368+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028368+0000","last_undegraded":"2024-04-26T19:51:42.028368+0000","last_fullsized":"2024-04-26T19:51:42.028368+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030071+0000","last_change":"2024-04-26T19:51:42.030071+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030071+0000","last_undegraded":"2024-04-26T19:51:42.030071+0000","last_fullsized":"2024-04-26T19:51:42.030071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028097+0000","last_change":"2024-04-26T19:51:42.028097+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028097+0000","last_undegraded":"2024-04-26T19:51:42.028097+0000","last_fullsized":"2024-04-26T19:51:42.028097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028936+0000","last_change":"2024-04-26T19:51:42.028936+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028936+0000","last_undegraded":"2024-04-26T19:51:42.028936+0000","last_fullsized":"2024-04-26T19:51:42.028936+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028210+0000","last_change":"2024-04-26T19:51:42.028210+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028210+0000","last_undegraded":"2024-04-26T19:51:42.028210+0000","last_fullsized":"2024-04-26T19:51:42.028210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.026285+0000","last_change":"2024-04-26T19:51:42.026285+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.026285+0000","last_undegraded":"2024-04-26T19:51:42.026285+0000","last_fullsized":"2024-04-26T19:51:42.026285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028023+0000","last_change":"2024-04-26T19:51:42.028023+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028023+0000","last_undegraded":"2024-04-26T19:51:42.028023+0000","last_fullsized":"2024-04-26T19:51:42.028023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033795+0000","last_change":"2024-04-26T19:51:42.033795+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033795+0000","last_undegraded":"2024-04-26T19:51:42.033795+0000","last_fullsized":"2024-04-26T19:51:42.033795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031632+0000","last_change":"2024-04-26T19:51:42.031632+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031632+0000","last_undegraded":"2024-04-26T19:51:42.031632+0000","last_fullsized":"2024-04-26T19:51:42.031632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T19:51:42.032692+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:51:42.032692+0000","last_peered":"2024-04-26T19:51:42.032692+0000","last_clean":"2024-04-26T19:51:42.032692+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:51:42.032692+0000","last_undegraded":"2024-04-26T19:51:42.032692+0000","last_fullsized":"2024-04-26T19:51:42.032692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:51:47.001 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:51:47.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.132+0000 7f7797499640 1 Processor -- start 2024-04-26T19:51:47.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.133+0000 7f7797499640 1 -- start start 2024-04-26T19:51:47.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.133+0000 7f7797499640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f779006ff00 0x7f7790148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.133+0000 7f7797499640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f77900d2cd0 0x7f77900d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.133+0000 7f7797499640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77900b1290 con 0x7f779006fb20 2024-04-26T19:51:47.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.133+0000 7f7797499640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f77900d3e60 con 0x7f77900d2cd0 2024-04-26T19:51:47.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.133+0000 7f7797499640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f77900d3110 con 0x7f779006ff00 2024-04-26T19:51:47.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.133+0000 7f7795235640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f779006fb20 0x7f77901488c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:57256/0 (socket says 172.21.15.79:57256) 2024-04-26T19:51:47.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.133+0000 7f7795235640 1 -- 172.21.15.79:0/3477384538 learned_addr learned my addr 172.21.15.79:0/3477384538 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:47.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7794a34640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f779006ff00 0x7f7790148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.137 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7795a36640 1 --2- 172.21.15.79:0/3477384538 >> v2:172.21.15.83:3300/0 conn(0x7f77900d2cd0 0x7f77900d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 335503348 0 0) 0x7f77900b1290 con 0x7f779006fb20 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f777c003520 con 0x7f779006fb20 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 988058721 0 0) 0x7f777c003520 con 0x7f779006fb20 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f779006ff00 msgr2=0x7f7790148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f779006ff00 0x7f7790148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 >> v2:172.21.15.83:3300/0 conn(0x7f77900d2cd0 msgr2=0x7f77900d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 --2- 172.21.15.79:0/3477384538 >> v2:172.21.15.83:3300/0 conn(0x7f77900d2cd0 0x7f77900d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77900d4090 con 0x7f779006fb20 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f778c0019f0 con 0x7f779006fb20 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 >> v1:172.21.15.79:6789/0 conn(0x7f779006fb20 legacy=0x7f77901488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.138 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f777c0039b0 0x7f777c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777c0043a0 0x7f777c004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f777c004ef0 0x7f777c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77900d3110 con 0x7f777c004ef0 2024-04-26T19:51:47.139 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f777c00bf00 con 0x7f777c0043a0 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f777c00c080 con 0x7f777c0039b0 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7795a36640 1 --2- 172.21.15.79:0/3477384538 >> v2:172.21.15.83:3300/0 conn(0x7f77900d2cd0 0x7f77900d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7794a34640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f779006ff00 0x7f7790148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7795a36640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f777c004ef0 0x7f777c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7795235640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f777c0039b0 0x7f777c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7794a34640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777c0043a0 0x7f777c004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7795a36640 1 -- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f777c0039b0 msgr2=0x7f777c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7795a36640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f777c0039b0 0x7f777c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7795a36640 1 -- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777c0043a0 msgr2=0x7f777c004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7795a36640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f777c0043a0 0x7f777c004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.134+0000 7f7795a36640 1 -- 172.21.15.79:0/3477384538 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c003520 con 0x7f777c004ef0 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.135+0000 7f7795a36640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f777c004ef0 0x7f777c00b9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7788009ca0 tx=0x7f777c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5aa8057630b702 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.135+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788009840 con 0x7f777c004ef0 2024-04-26T19:51:47.140 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.135+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7788004020 con 0x7f777c004ef0 2024-04-26T19:51:47.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.135+0000 7f7787fff640 1 -- 172.21.15.79:0/3477384538 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880043d0 con 0x7f777c004ef0 2024-04-26T19:51:47.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.136+0000 7f7797499640 1 -- 172.21.15.79:0/3477384538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f777c004ef0 msgr2=0x7f777c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.136+0000 7f7797499640 1 --2- 172.21.15.79:0/3477384538 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f777c004ef0 0x7f777c00b9c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7788009ca0 tx=0x7f777c005830 comp rx=0 tx=0).stop 2024-04-26T19:51:47.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.136+0000 7f7795a36640 1 -- 172.21.15.79:0/3477384538 reap_dead start 2024-04-26T19:51:47.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.136+0000 7f7797499640 1 -- 172.21.15.79:0/3477384538 shutdown_connections 2024-04-26T19:51:47.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.136+0000 7f7797499640 1 -- 172.21.15.79:0/3477384538 >> 172.21.15.79:0/3477384538 conn(0x7f7790071c30 msgr2=0x7f779006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:47.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.136+0000 7f7797499640 1 -- 172.21.15.79:0/3477384538 shutdown_connections 2024-04-26T19:51:47.141 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.136+0000 7f7797499640 1 -- 172.21.15.79:0/3477384538 wait complete. 2024-04-26T19:51:47.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.137+0000 7f7797499640 1 Processor -- start 2024-04-26T19:51:47.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.137+0000 7f7797499640 1 -- start start 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7797499640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7790073000 0x7f77900c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7797499640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77900733e0 0x7f7790161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7797499640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7790155350 0x7f77901557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7797499640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f77900d4930 con 0x7f77900733e0 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7797499640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f77900d4650 con 0x7f7790155350 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7797499640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f77900d4c10 con 0x7f7790073000 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7794a34640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77900733e0 0x7f7790161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7794a34640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77900733e0 0x7f7790161cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54184/0 (socket says 172.21.15.79:54184) 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7794a34640 1 -- 172.21.15.79:0/4122864137 learned_addr learned my addr 172.21.15.79:0/4122864137 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7794a34640 1 -- 172.21.15.79:0/4122864137 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7790073000 msgr2=0x7f77900c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7794a34640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7790073000 0x7f77900c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7794a34640 1 -- 172.21.15.79:0/4122864137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7790155350 msgr2=0x7f77901557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7794a34640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7790155350 0x7f77901557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7794a34640 1 -- 172.21.15.79:0/4122864137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77900c5da0 con 0x7f77900733e0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7794a34640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77900733e0 0x7f7790161cf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7780011800 tx=0x7f7780011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7785ffb640 1 -- 172.21.15.79:0/4122864137 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778000f940 con 0x7f77900733e0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7785ffb640 1 -- 172.21.15.79:0/4122864137 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778000fae0 con 0x7f77900733e0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.138+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77900c6080 con 0x7f77900733e0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.139+0000 7f7785ffb640 1 -- 172.21.15.79:0/4122864137 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7780015460 con 0x7f77900733e0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.139+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77900c6640 con 0x7f77900733e0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.140+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77900506c0 con 0x7f77900733e0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.140+0000 7f7785ffb640 1 -- 172.21.15.79:0/4122864137 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7780015600 con 0x7f77900733e0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.141+0000 7f7785ffb640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f777406c920 0x7f777406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.141+0000 7f7785ffb640 1 -- 172.21.15.79:0/4122864137 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f778008c630 con 0x7f77900733e0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.141+0000 7f7795235640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f777406c920 0x7f777406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.141+0000 7f7795235640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f777406c920 0x7f777406ede0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f778c0071f0 tx=0x7f778c001270 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:47.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.145+0000 7f7785ffb640 1 -- 172.21.15.79:0/4122864137 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7780019030 con 0x7f77900733e0 2024-04-26T19:51:47.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.316+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7790162230 con 0x7f777406c920 2024-04-26T19:51:47.322 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.322+0000 7f7785ffb640 1 -- 172.21.15.79:0/4122864137 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240051 (secure 0 0 0) 0x7f778c007a20 con 0x7f777406c920 2024-04-26T19:51:47.324 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:47.326 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:51:47.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f777406c920 msgr2=0x7f777406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f777406c920 0x7f777406ede0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f778c0071f0 tx=0x7f778c001270 comp rx=0 tx=0).stop 2024-04-26T19:51:47.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77900733e0 msgr2=0x7f7790161cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77900733e0 0x7f7790161cf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7780011800 tx=0x7f7780011cc0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 shutdown_connections 2024-04-26T19:51:47.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f777406c920 0x7f777406ede0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7790155350 0x7f77901557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f77900733e0 0x7f7790161cf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 --2- 172.21.15.79:0/4122864137 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7790073000 0x7f77900c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 >> 172.21.15.79:0/4122864137 conn(0x7f7790071c30 msgr2=0x7f7790072040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:47.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 shutdown_connections 2024-04-26T19:51:47.328 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.325+0000 7f7797499640 1 -- 172.21.15.79:0/4122864137 wait complete. 2024-04-26T19:51:47.336 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-26T19:51:46.271542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220512,"kb_used_data":4332,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4435968,"data_stored":2562739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.246429"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.026344+0000","last_change":"2024-04-26T19:51:42.026344+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.026344+0000","last_undegraded":"2024-04-26T19:51:42.026344+0000","last_fullsized":"2024-04-26T19:51:42.026344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028843+0000","last_change":"2024-04-26T19:51:42.028843+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028843+0000","last_undegraded":"2024-04-26T19:51:42.028843+0000","last_fullsized":"2024-04-26T19:51:42.028843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031161+0000","last_change":"2024-04-26T19:51:42.031161+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031161+0000","last_undegraded":"2024-04-26T19:51:42.031161+0000","last_fullsized":"2024-04-26T19:51:42.031161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028023+0000","last_change":"2024-04-26T19:51:42.028023+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028023+0000","last_undegraded":"2024-04-26T19:51:42.028023+0000","last_fullsized":"2024-04-26T19:51:42.028023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030077+0000","last_change":"2024-04-26T19:51:42.030077+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030077+0000","last_undegraded":"2024-04-26T19:51:42.030077+0000","last_fullsized":"2024-04-26T19:51:42.030077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031054+0000","last_change":"2024-04-26T19:51:42.031054+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031054+0000","last_undegraded":"2024-04-26T19:51:42.031054+0000","last_fullsized":"2024-04-26T19:51:42.031054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031173+0000","last_change":"2024-04-26T19:51:42.031173+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031173+0000","last_undegraded":"2024-04-26T19:51:42.031173+0000","last_fullsized":"2024-04-26T19:51:42.031173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028030+0000","last_change":"2024-04-26T19:51:42.028030+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028030+0000","last_undegraded":"2024-04-26T19:51:42.028030+0000","last_fullsized":"2024-04-26T19:51:42.028030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033452+0000","last_change":"2024-04-26T19:51:42.033452+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033452+0000","last_undegraded":"2024-04-26T19:51:42.033452+0000","last_fullsized":"2024-04-26T19:51:42.033452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030122+0000","last_change":"2024-04-26T19:51:42.030122+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030122+0000","last_undegraded":"2024-04-26T19:51:42.030122+0000","last_fullsized":"2024-04-26T19:51:42.030122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030131+0000","last_change":"2024-04-26T19:51:42.030131+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030131+0000","last_undegraded":"2024-04-26T19:51:42.030131+0000","last_fullsized":"2024-04-26T19:51:42.030131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033525+0000","last_change":"2024-04-26T19:51:42.033525+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033525+0000","last_undegraded":"2024-04-26T19:51:42.033525+0000","last_fullsized":"2024-04-26T19:51:42.033525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033417+0000","last_change":"2024-04-26T19:51:42.033417+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033417+0000","last_undegraded":"2024-04-26T19:51:42.033417+0000","last_fullsized":"2024-04-26T19:51:42.033417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031159+0000","last_change":"2024-04-26T19:51:42.031159+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031159+0000","last_undegraded":"2024-04-26T19:51:42.031159+0000","last_fullsized":"2024-04-26T19:51:42.031159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.026344+0000","last_change":"2024-04-26T19:51:42.026344+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.026344+0000","last_undegraded":"2024-04-26T19:51:42.026344+0000","last_fullsized":"2024-04-26T19:51:42.026344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.032485+0000","last_change":"2024-04-26T19:51:42.032485+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.032485+0000","last_undegraded":"2024-04-26T19:51:42.032485+0000","last_fullsized":"2024-04-26T19:51:42.032485+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030545+0000","last_change":"2024-04-26T19:51:42.030545+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030545+0000","last_undegraded":"2024-04-26T19:51:42.030545+0000","last_fullsized":"2024-04-26T19:51:42.030545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028035+0000","last_change":"2024-04-26T19:51:42.028035+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028035+0000","last_undegraded":"2024-04-26T19:51:42.028035+0000","last_fullsized":"2024-04-26T19:51:42.028035+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028448+0000","last_change":"2024-04-26T19:51:42.028448+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028448+0000","last_undegraded":"2024-04-26T19:51:42.028448+0000","last_fullsized":"2024-04-26T19:51:42.028448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031161+0000","last_change":"2024-04-26T19:51:42.031161+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031161+0000","last_undegraded":"2024-04-26T19:51:42.031161+0000","last_fullsized":"2024-04-26T19:51:42.031161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028450+0000","last_change":"2024-04-26T19:51:42.028450+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028450+0000","last_undegraded":"2024-04-26T19:51:42.028450+0000","last_fullsized":"2024-04-26T19:51:42.028450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028357+0000","last_change":"2024-04-26T19:51:42.028357+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028357+0000","last_undegraded":"2024-04-26T19:51:42.028357+0000","last_fullsized":"2024-04-26T19:51:42.028357+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031981+0000","last_change":"2024-04-26T19:51:42.031981+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031981+0000","last_undegraded":"2024-04-26T19:51:42.031981+0000","last_fullsized":"2024-04-26T19:51:42.031981+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028368+0000","last_change":"2024-04-26T19:51:42.028368+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028368+0000","last_undegraded":"2024-04-26T19:51:42.028368+0000","last_fullsized":"2024-04-26T19:51:42.028368+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030071+0000","last_change":"2024-04-26T19:51:42.030071+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030071+0000","last_undegraded":"2024-04-26T19:51:42.030071+0000","last_fullsized":"2024-04-26T19:51:42.030071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028097+0000","last_change":"2024-04-26T19:51:42.028097+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028097+0000","last_undegraded":"2024-04-26T19:51:42.028097+0000","last_fullsized":"2024-04-26T19:51:42.028097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028936+0000","last_change":"2024-04-26T19:51:42.028936+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028936+0000","last_undegraded":"2024-04-26T19:51:42.028936+0000","last_fullsized":"2024-04-26T19:51:42.028936+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028210+0000","last_change":"2024-04-26T19:51:42.028210+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028210+0000","last_undegraded":"2024-04-26T19:51:42.028210+0000","last_fullsized":"2024-04-26T19:51:42.028210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.026285+0000","last_change":"2024-04-26T19:51:42.026285+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.026285+0000","last_undegraded":"2024-04-26T19:51:42.026285+0000","last_fullsized":"2024-04-26T19:51:42.026285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028023+0000","last_change":"2024-04-26T19:51:42.028023+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028023+0000","last_undegraded":"2024-04-26T19:51:42.028023+0000","last_fullsized":"2024-04-26T19:51:42.028023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033795+0000","last_change":"2024-04-26T19:51:42.033795+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033795+0000","last_undegraded":"2024-04-26T19:51:42.033795+0000","last_fullsized":"2024-04-26T19:51:42.033795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031632+0000","last_change":"2024-04-26T19:51:42.031632+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031632+0000","last_undegraded":"2024-04-26T19:51:42.031632+0000","last_fullsized":"2024-04-26T19:51:42.031632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T19:51:42.032692+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:51:42.032692+0000","last_peered":"2024-04-26T19:51:42.032692+0000","last_clean":"2024-04-26T19:51:42.032692+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:51:42.032692+0000","last_undegraded":"2024-04-26T19:51:42.032692+0000","last_fullsized":"2024-04-26T19:51:42.032692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:51:47.340 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:51:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.472+0000 7fea02e37640 1 Processor -- start 2024-04-26T19:51:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.472+0000 7fea02e37640 1 -- start start 2024-04-26T19:51:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.473+0000 7fea02e37640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe9fc163020 0x7fe9fc163460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.473+0000 7fea02e37640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fe9fc071d60 0x7fe9fc0721b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.473+0000 7fea02e37640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9fc06f810 con 0x7fe9fc1422c0 2024-04-26T19:51:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.473+0000 7fea02e37640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fe9fc0718d0 con 0x7fe9fc071d60 2024-04-26T19:51:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.473+0000 7fea02e37640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe9fc1639a0 con 0x7fe9fc163020 2024-04-26T19:51:47.476 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.473+0000 7fea00bd3640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fe9fc1422c0 0x7fe9fc06f100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53608/0 (socket says 172.21.15.79:53608) 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.473+0000 7fea00bd3640 1 -- 172.21.15.79:0/1843616827 learned_addr learned my addr 172.21.15.79:0/1843616827 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.473+0000 7fea013d4640 1 --2- 172.21.15.79:0/1843616827 >> v2:172.21.15.79:3300/0 conn(0x7fe9fc071d60 0x7fe9fc0721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.473+0000 7fe9fbfff640 1 --2- 172.21.15.79:0/1843616827 >> v2:172.21.15.83:3300/0 conn(0x7fe9fc163020 0x7fe9fc163460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fea013d4640 1 -- 172.21.15.79:0/1843616827 >> v2:172.21.15.83:3300/0 conn(0x7fe9fc163020 msgr2=0x7fe9fc163460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fea013d4640 1 --2- 172.21.15.79:0/1843616827 >> v2:172.21.15.83:3300/0 conn(0x7fe9fc163020 0x7fe9fc163460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fea013d4640 1 -- 172.21.15.79:0/1843616827 >> v1:172.21.15.79:6789/0 conn(0x7fe9fc1422c0 legacy=0x7fe9fc06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fe9fb7fe640 1 -- 172.21.15.79:0/1843616827 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2864719729 0 0) 0x7fe9fc06f810 con 0x7fe9fc1422c0 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fea013d4640 1 -- 172.21.15.79:0/1843616827 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc06fac0 con 0x7fe9fc071d60 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fe9fbfff640 1 --2- 172.21.15.79:0/1843616827 >> v2:172.21.15.83:3300/0 conn(0x7fe9fc163020 0x7fe9fc163460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fea013d4640 1 --2- 172.21.15.79:0/1843616827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc071d60 0x7fe9fc0721b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc06eeb0 tx=0x7fe9f402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d56c91283fb48a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fe9fb7fe640 1 -- 172.21.15.79:0/1843616827 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4007740 con 0x7fe9fc071d60 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fe9fb7fe640 1 -- 172.21.15.79:0/1843616827 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f40078e0 con 0x7fe9fc071d60 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.474+0000 7fe9fb7fe640 1 -- 172.21.15.79:0/1843616827 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4007cb0 con 0x7fe9fc071d60 2024-04-26T19:51:47.477 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.475+0000 7fea02e37640 1 -- 172.21.15.79:0/1843616827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc071d60 msgr2=0x7fe9fc0721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.475+0000 7fea02e37640 1 --2- 172.21.15.79:0/1843616827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc071d60 0x7fe9fc0721b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc06eeb0 tx=0x7fe9f402a5f0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.475+0000 7fea02e37640 1 -- 172.21.15.79:0/1843616827 shutdown_connections 2024-04-26T19:51:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.475+0000 7fea02e37640 1 --2- 172.21.15.79:0/1843616827 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc071d60 0x7fe9fc0721b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.475+0000 7fea02e37640 1 --2- 172.21.15.79:0/1843616827 >> v2:172.21.15.83:3300/0 conn(0x7fe9fc163020 0x7fe9fc163460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.475+0000 7fea02e37640 1 -- 172.21.15.79:0/1843616827 >> 172.21.15.79:0/1843616827 conn(0x7fe9fc0b2a70 msgr2=0x7fe9fc0b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.475+0000 7fea02e37640 1 -- 172.21.15.79:0/1843616827 shutdown_connections 2024-04-26T19:51:47.478 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.475+0000 7fea02e37640 1 -- 172.21.15.79:0/1843616827 wait complete. 2024-04-26T19:51:47.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.476+0000 7fea02e37640 1 Processor -- start 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.476+0000 7fea02e37640 1 -- start start 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fea02e37640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe9fc071d60 0x7fe9fc156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fea02e37640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc1422c0 0x7fe9fc1f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fea02e37640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9fc163020 0x7fe9fc1f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fea02e37640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc070360 con 0x7fe9fc1422c0 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fea02e37640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc070080 con 0x7fe9fc163020 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fea02e37640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc0ae8d0 con 0x7fe9fc071d60 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fe9fbfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc1422c0 0x7fe9fc1f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fe9fbfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc1422c0 0x7fe9fc1f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49004/0 (socket says 172.21.15.79:49004) 2024-04-26T19:51:47.481 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fe9fbfff640 1 -- 172.21.15.79:0/125218036 learned_addr learned my addr 172.21.15.79:0/125218036 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fea013d4640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9fc163020 0x7fe9fc1f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fea00bd3640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe9fc071d60 0x7fe9fc156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fe9fbfff640 1 -- 172.21.15.79:0/125218036 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe9fc071d60 msgr2=0x7fe9fc156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fe9fbfff640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe9fc071d60 0x7fe9fc156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fe9fbfff640 1 -- 172.21.15.79:0/125218036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9fc163020 msgr2=0x7fe9fc1f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fe9fbfff640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9fc163020 0x7fe9fc1f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fe9fbfff640 1 -- 172.21.15.79:0/125218036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc148bc0 con 0x7fe9fc1422c0 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.477+0000 7fea013d4640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9fc163020 0x7fe9fc1f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.478+0000 7fe9fbfff640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc1422c0 0x7fe9fc1f8290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000d7d0 tx=0x7fe9f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.478+0000 7fe9f97fa640 1 -- 172.21.15.79:0/125218036 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0004030 con 0x7fe9fc1422c0 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.478+0000 7fe9f97fa640 1 -- 172.21.15.79:0/125218036 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f00041d0 con 0x7fe9fc1422c0 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.478+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc145f30 con 0x7fe9fc1422c0 2024-04-26T19:51:47.482 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.478+0000 7fe9f97fa640 1 -- 172.21.15.79:0/125218036 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f00096a0 con 0x7fe9fc1422c0 2024-04-26T19:51:47.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.478+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc1464f0 con 0x7fe9fc1422c0 2024-04-26T19:51:47.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.479+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc0506c0 con 0x7fe9fc1422c0 2024-04-26T19:51:47.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.479+0000 7fe9f97fa640 1 -- 172.21.15.79:0/125218036 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9f0009840 con 0x7fe9fc1422c0 2024-04-26T19:51:47.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.480+0000 7fe9f97fa640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe9e806c8f0 0x7fe9e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.480+0000 7fe9f97fa640 1 -- 172.21.15.79:0/125218036 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe9f008bf40 con 0x7fe9fc1422c0 2024-04-26T19:51:47.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.480+0000 7fea00bd3640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe9e806c8f0 0x7fe9e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.483 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.480+0000 7fea00bd3640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe9e806c8f0 0x7fe9e806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000f60 tx=0x7fe9ec007b50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:47.484 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.484+0000 7fe9f97fa640 1 -- 172.21.15.79:0/125218036 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe9f005a270 con 0x7fe9fc1422c0 2024-04-26T19:51:47.654 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.654+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9fc069820 con 0x7fe9e806c8f0 2024-04-26T19:51:47.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.659+0000 7fe9f97fa640 1 -- 172.21.15.79:0/125218036 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240051 (secure 0 0 0) 0x7fe9fc069820 con 0x7fe9e806c8f0 2024-04-26T19:51:47.663 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:47.664 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe9e806c8f0 msgr2=0x7fe9e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe9e806c8f0 0x7fe9e806edb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec000f60 tx=0x7fe9ec007b50 comp rx=0 tx=0).stop 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc1422c0 msgr2=0x7fe9fc1f8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc1422c0 0x7fe9fc1f8290 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000d7d0 tx=0x7fe9f000dc90 comp rx=0 tx=0).stop 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 shutdown_connections 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fe9e806c8f0 0x7fe9e806edb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe9fc163020 0x7fe9fc1f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe9fc1422c0 0x7fe9fc1f8290 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 --2- 172.21.15.79:0/125218036 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe9fc071d60 0x7fe9fc156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 >> 172.21.15.79:0/125218036 conn(0x7fe9fc0b2a70 msgr2=0x7fe9fc160fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:47.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 shutdown_connections 2024-04-26T19:51:47.668 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.663+0000 7fea02e37640 1 -- 172.21.15.79:0/125218036 wait complete. 2024-04-26T19:51:47.675 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-04-26T19:51:46.271542+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220512,"kb_used_data":4332,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754208,"statfs":{"total":773094113280,"available":772868308992,"internally_reserved":0,"allocated":4435968,"data_stored":2562739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.246429"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.026344+0000","last_change":"2024-04-26T19:51:42.026344+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.026344+0000","last_undegraded":"2024-04-26T19:51:42.026344+0000","last_fullsized":"2024-04-26T19:51:42.026344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028843+0000","last_change":"2024-04-26T19:51:42.028843+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028843+0000","last_undegraded":"2024-04-26T19:51:42.028843+0000","last_fullsized":"2024-04-26T19:51:42.028843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031161+0000","last_change":"2024-04-26T19:51:42.031161+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031161+0000","last_undegraded":"2024-04-26T19:51:42.031161+0000","last_fullsized":"2024-04-26T19:51:42.031161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028023+0000","last_change":"2024-04-26T19:51:42.028023+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028023+0000","last_undegraded":"2024-04-26T19:51:42.028023+0000","last_fullsized":"2024-04-26T19:51:42.028023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030077+0000","last_change":"2024-04-26T19:51:42.030077+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030077+0000","last_undegraded":"2024-04-26T19:51:42.030077+0000","last_fullsized":"2024-04-26T19:51:42.030077+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031054+0000","last_change":"2024-04-26T19:51:42.031054+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031054+0000","last_undegraded":"2024-04-26T19:51:42.031054+0000","last_fullsized":"2024-04-26T19:51:42.031054+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031173+0000","last_change":"2024-04-26T19:51:42.031173+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031173+0000","last_undegraded":"2024-04-26T19:51:42.031173+0000","last_fullsized":"2024-04-26T19:51:42.031173+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028030+0000","last_change":"2024-04-26T19:51:42.028030+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028030+0000","last_undegraded":"2024-04-26T19:51:42.028030+0000","last_fullsized":"2024-04-26T19:51:42.028030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033452+0000","last_change":"2024-04-26T19:51:42.033452+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033452+0000","last_undegraded":"2024-04-26T19:51:42.033452+0000","last_fullsized":"2024-04-26T19:51:42.033452+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030122+0000","last_change":"2024-04-26T19:51:42.030122+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030122+0000","last_undegraded":"2024-04-26T19:51:42.030122+0000","last_fullsized":"2024-04-26T19:51:42.030122+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030131+0000","last_change":"2024-04-26T19:51:42.030131+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030131+0000","last_undegraded":"2024-04-26T19:51:42.030131+0000","last_fullsized":"2024-04-26T19:51:42.030131+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033525+0000","last_change":"2024-04-26T19:51:42.033525+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033525+0000","last_undegraded":"2024-04-26T19:51:42.033525+0000","last_fullsized":"2024-04-26T19:51:42.033525+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033417+0000","last_change":"2024-04-26T19:51:42.033417+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033417+0000","last_undegraded":"2024-04-26T19:51:42.033417+0000","last_fullsized":"2024-04-26T19:51:42.033417+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031159+0000","last_change":"2024-04-26T19:51:42.031159+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031159+0000","last_undegraded":"2024-04-26T19:51:42.031159+0000","last_fullsized":"2024-04-26T19:51:42.031159+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.026344+0000","last_change":"2024-04-26T19:51:42.026344+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.026344+0000","last_undegraded":"2024-04-26T19:51:42.026344+0000","last_fullsized":"2024-04-26T19:51:42.026344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.032485+0000","last_change":"2024-04-26T19:51:42.032485+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.032485+0000","last_undegraded":"2024-04-26T19:51:42.032485+0000","last_fullsized":"2024-04-26T19:51:42.032485+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030545+0000","last_change":"2024-04-26T19:51:42.030545+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030545+0000","last_undegraded":"2024-04-26T19:51:42.030545+0000","last_fullsized":"2024-04-26T19:51:42.030545+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028035+0000","last_change":"2024-04-26T19:51:42.028035+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028035+0000","last_undegraded":"2024-04-26T19:51:42.028035+0000","last_fullsized":"2024-04-26T19:51:42.028035+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028448+0000","last_change":"2024-04-26T19:51:42.028448+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028448+0000","last_undegraded":"2024-04-26T19:51:42.028448+0000","last_fullsized":"2024-04-26T19:51:42.028448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031161+0000","last_change":"2024-04-26T19:51:42.031161+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031161+0000","last_undegraded":"2024-04-26T19:51:42.031161+0000","last_fullsized":"2024-04-26T19:51:42.031161+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028450+0000","last_change":"2024-04-26T19:51:42.028450+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028450+0000","last_undegraded":"2024-04-26T19:51:42.028450+0000","last_fullsized":"2024-04-26T19:51:42.028450+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028357+0000","last_change":"2024-04-26T19:51:42.028357+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028357+0000","last_undegraded":"2024-04-26T19:51:42.028357+0000","last_fullsized":"2024-04-26T19:51:42.028357+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031981+0000","last_change":"2024-04-26T19:51:42.031981+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031981+0000","last_undegraded":"2024-04-26T19:51:42.031981+0000","last_fullsized":"2024-04-26T19:51:42.031981+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028368+0000","last_change":"2024-04-26T19:51:42.028368+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028368+0000","last_undegraded":"2024-04-26T19:51:42.028368+0000","last_fullsized":"2024-04-26T19:51:42.028368+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.030071+0000","last_change":"2024-04-26T19:51:42.030071+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.030071+0000","last_undegraded":"2024-04-26T19:51:42.030071+0000","last_fullsized":"2024-04-26T19:51:42.030071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028097+0000","last_change":"2024-04-26T19:51:42.028097+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028097+0000","last_undegraded":"2024-04-26T19:51:42.028097+0000","last_fullsized":"2024-04-26T19:51:42.028097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028936+0000","last_change":"2024-04-26T19:51:42.028936+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028936+0000","last_undegraded":"2024-04-26T19:51:42.028936+0000","last_fullsized":"2024-04-26T19:51:42.028936+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028210+0000","last_change":"2024-04-26T19:51:42.028210+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028210+0000","last_undegraded":"2024-04-26T19:51:42.028210+0000","last_fullsized":"2024-04-26T19:51:42.028210+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.026285+0000","last_change":"2024-04-26T19:51:42.026285+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.026285+0000","last_undegraded":"2024-04-26T19:51:42.026285+0000","last_fullsized":"2024-04-26T19:51:42.026285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.028023+0000","last_change":"2024-04-26T19:51:42.028023+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.028023+0000","last_undegraded":"2024-04-26T19:51:42.028023+0000","last_fullsized":"2024-04-26T19:51:42.028023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.033795+0000","last_change":"2024-04-26T19:51:42.033795+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.033795+0000","last_undegraded":"2024-04-26T19:51:42.033795+0000","last_fullsized":"2024-04-26T19:51:42.033795+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-26T19:51:43.022971+0000","last_change":"2024-04-26T19:51:43.022971+0000","last_active":"2024-04-26T19:51:43.022971+0000","last_peered":"2024-04-26T19:51:43.022971+0000","last_clean":"2024-04-26T19:51:43.022971+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:43.022971+0000","last_undegraded":"2024-04-26T19:51:43.022971+0000","last_fullsized":"2024-04-26T19:51:43.022971+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":36,"state":"creating+peering","last_fresh":"2024-04-26T19:51:42.031632+0000","last_change":"2024-04-26T19:51:42.031632+0000","last_active":"2024-04-26T19:51:42.019054+0000","last_peered":"2024-04-26T19:51:42.019054+0000","last_clean":"2024-04-26T19:51:42.019054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T19:51:42.031632+0000","last_undegraded":"2024-04-26T19:51:42.031632+0000","last_fullsized":"2024-04-26T19:51:42.031632+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":199,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-04-26T19:51:42.032692+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:51:42.032692+0000","last_peered":"2024-04-26T19:51:42.032692+0000","last_clean":"2024-04-26T19:51:42.032692+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:51:42.032692+0000","last_undegraded":"2024-04-26T19:51:42.032692+0000","last_fullsized":"2024-04-26T19:51:42.032692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":413696,"data_stored":176631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":401408,"data_stored":170422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":766999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:51:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-04-26T19:51:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-26T19:51:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-04-26T19:51:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-26T19:51:47.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-04-26T19:51:47.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-04-26T19:51:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-26T19:51:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-26T19:51:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-04-26T19:51:47.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-04-26T19:51:47.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-26T19:51:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-04-26T19:51:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-04-26T19:51:47.685 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-26T19:51:47.686 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.026344+0000', 'last_change': '2024-04-26T19:51:42.026344+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.026344+0000', 'last_undegraded': '2024-04-26T19:51:42.026344+0000', 'last_fullsized': '2024-04-26T19:51:42.026344+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028843+0000', 'last_change': '2024-04-26T19:51:42.028843+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028843+0000', 'last_undegraded': '2024-04-26T19:51:42.028843+0000', 'last_fullsized': '2024-04-26T19:51:42.028843+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.031161+0000', 'last_change': '2024-04-26T19:51:42.031161+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.031161+0000', 'last_undegraded': '2024-04-26T19:51:42.031161+0000', 'last_fullsized': '2024-04-26T19:51:42.031161+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T19:51:47.687 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028023+0000', 'last_change': '2024-04-26T19:51:42.028023+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028023+0000', 'last_undegraded': '2024-04-26T19:51:42.028023+0000', 'last_fullsized': '2024-04-26T19:51:42.028023+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.030077+0000', 'last_change': '2024-04-26T19:51:42.030077+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.030077+0000', 'last_undegraded': '2024-04-26T19:51:42.030077+0000', 'last_fullsized': '2024-04-26T19:51:42.030077+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-26T19:51:47.688 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.031054+0000', 'last_change': '2024-04-26T19:51:42.031054+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.031054+0000', 'last_undegraded': '2024-04-26T19:51:42.031054+0000', 'last_fullsized': '2024-04-26T19:51:42.031054+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.031173+0000', 'last_change': '2024-04-26T19:51:42.031173+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.031173+0000', 'last_undegraded': '2024-04-26T19:51:42.031173+0000', 'last_fullsized': '2024-04-26T19:51:42.031173+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028030+0000', 'last_change': '2024-04-26T19:51:42.028030+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028030+0000', 'last_undegraded': '2024-04-26T19:51:42.028030+0000', 'last_fullsized': '2024-04-26T19:51:42.028030+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-26T19:51:47.689 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.033452+0000', 'last_change': '2024-04-26T19:51:42.033452+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.033452+0000', 'last_undegraded': '2024-04-26T19:51:42.033452+0000', 'last_fullsized': '2024-04-26T19:51:42.033452+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.030122+0000', 'last_change': '2024-04-26T19:51:42.030122+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.030122+0000', 'last_undegraded': '2024-04-26T19:51:42.030122+0000', 'last_fullsized': '2024-04-26T19:51:42.030122+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.030131+0000', 'last_change': '2024-04-26T19:51:42.030131+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.030131+0000', 'last_undegraded': '2024-04-26T19:51:42.030131+0000', 'last_fullsized': '2024-04-26T19:51:42.030131+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T19:51:47.690 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.033525+0000', 'last_change': '2024-04-26T19:51:42.033525+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.033525+0000', 'last_undegraded': '2024-04-26T19:51:42.033525+0000', 'last_fullsized': '2024-04-26T19:51:42.033525+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.033417+0000', 'last_change': '2024-04-26T19:51:42.033417+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.033417+0000', 'last_undegraded': '2024-04-26T19:51:42.033417+0000', 'last_fullsized': '2024-04-26T19:51:42.033417+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-26T19:51:47.691 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.031159+0000', 'last_change': '2024-04-26T19:51:42.031159+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.031159+0000', 'last_undegraded': '2024-04-26T19:51:42.031159+0000', 'last_fullsized': '2024-04-26T19:51:42.031159+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.026344+0000', 'last_change': '2024-04-26T19:51:42.026344+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.026344+0000', 'last_undegraded': '2024-04-26T19:51:42.026344+0000', 'last_fullsized': '2024-04-26T19:51:42.026344+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.032485+0000', 'last_change': '2024-04-26T19:51:42.032485+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.032485+0000', 'last_undegraded': '2024-04-26T19:51:42.032485+0000', 'last_fullsized': '2024-04-26T19:51:42.032485+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-26T19:51:47.692 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.030545+0000', 'last_change': '2024-04-26T19:51:42.030545+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.030545+0000', 'last_undegraded': '2024-04-26T19:51:42.030545+0000', 'last_fullsized': '2024-04-26T19:51:42.030545+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T19:51:47.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-26T19:51:47.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-26T19:51:47.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028035+0000', 'last_change': '2024-04-26T19:51:42.028035+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028035+0000', 'last_undegraded': '2024-04-26T19:51:42.028035+0000', 'last_fullsized': '2024-04-26T19:51:42.028035+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T19:51:47.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-26T19:51:47.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-26T19:51:47.693 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028448+0000', 'last_change': '2024-04-26T19:51:42.028448+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028448+0000', 'last_undegraded': '2024-04-26T19:51:42.028448+0000', 'last_fullsized': '2024-04-26T19:51:42.028448+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T19:51:47.693 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-26T19:51:47.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-26T19:51:47.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.031161+0000', 'last_change': '2024-04-26T19:51:42.031161+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.031161+0000', 'last_undegraded': '2024-04-26T19:51:42.031161+0000', 'last_fullsized': '2024-04-26T19:51:42.031161+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-26T19:51:47.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-26T19:51:47.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.694 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-26T19:51:47.694 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028450+0000', 'last_change': '2024-04-26T19:51:42.028450+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028450+0000', 'last_undegraded': '2024-04-26T19:51:42.028450+0000', 'last_fullsized': '2024-04-26T19:51:42.028450+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T19:51:47.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-26T19:51:47.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-26T19:51:47.695 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028357+0000', 'last_change': '2024-04-26T19:51:42.028357+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028357+0000', 'last_undegraded': '2024-04-26T19:51:42.028357+0000', 'last_fullsized': '2024-04-26T19:51:42.028357+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T19:51:47.695 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-26T19:51:47.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-26T19:51:47.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.031981+0000', 'last_change': '2024-04-26T19:51:42.031981+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.031981+0000', 'last_undegraded': '2024-04-26T19:51:42.031981+0000', 'last_fullsized': '2024-04-26T19:51:42.031981+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T19:51:47.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-26T19:51:47.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-26T19:51:47.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028368+0000', 'last_change': '2024-04-26T19:51:42.028368+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028368+0000', 'last_undegraded': '2024-04-26T19:51:42.028368+0000', 'last_fullsized': '2024-04-26T19:51:42.028368+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T19:51:47.696 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-26T19:51:47.696 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-26T19:51:47.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.030071+0000', 'last_change': '2024-04-26T19:51:42.030071+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.030071+0000', 'last_undegraded': '2024-04-26T19:51:42.030071+0000', 'last_fullsized': '2024-04-26T19:51:42.030071+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-26T19:51:47.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-26T19:51:47.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-26T19:51:47.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028097+0000', 'last_change': '2024-04-26T19:51:42.028097+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028097+0000', 'last_undegraded': '2024-04-26T19:51:42.028097+0000', 'last_fullsized': '2024-04-26T19:51:42.028097+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T19:51:47.697 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-26T19:51:47.697 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-26T19:51:47.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028936+0000', 'last_change': '2024-04-26T19:51:42.028936+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028936+0000', 'last_undegraded': '2024-04-26T19:51:42.028936+0000', 'last_fullsized': '2024-04-26T19:51:42.028936+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T19:51:47.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-26T19:51:47.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-26T19:51:47.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028210+0000', 'last_change': '2024-04-26T19:51:42.028210+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028210+0000', 'last_undegraded': '2024-04-26T19:51:42.028210+0000', 'last_fullsized': '2024-04-26T19:51:42.028210+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T19:51:47.698 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-26T19:51:47.698 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-26T19:51:47.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.026285+0000', 'last_change': '2024-04-26T19:51:42.026285+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.026285+0000', 'last_undegraded': '2024-04-26T19:51:42.026285+0000', 'last_fullsized': '2024-04-26T19:51:42.026285+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-26T19:51:47.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-26T19:51:47.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-26T19:51:47.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.028023+0000', 'last_change': '2024-04-26T19:51:42.028023+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.028023+0000', 'last_undegraded': '2024-04-26T19:51:42.028023+0000', 'last_fullsized': '2024-04-26T19:51:42.028023+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-26T19:51:47.699 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-26T19:51:47.699 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-26T19:51:47.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.033795+0000', 'last_change': '2024-04-26T19:51:42.033795+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.033795+0000', 'last_undegraded': '2024-04-26T19:51:42.033795+0000', 'last_fullsized': '2024-04-26T19:51:42.033795+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-26T19:51:47.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-26T19:51:47.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-26T19:51:43.022971+0000', 'last_change': '2024-04-26T19:51:43.022971+0000', 'last_active': '2024-04-26T19:51:43.022971+0000', 'last_peered': '2024-04-26T19:51:43.022971+0000', 'last_clean': '2024-04-26T19:51:43.022971+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:43.022971+0000', 'last_undegraded': '2024-04-26T19:51:43.022971+0000', 'last_fullsized': '2024-04-26T19:51:43.022971+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:43.022971+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-26T19:51:47.700 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-26T19:51:47.700 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 36, 'state': 'creating+peering', 'last_fresh': '2024-04-26T19:51:42.031632+0000', 'last_change': '2024-04-26T19:51:42.031632+0000', 'last_active': '2024-04-26T19:51:42.019054+0000', 'last_peered': '2024-04-26T19:51:42.019054+0000', 'last_clean': '2024-04-26T19:51:42.019054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-26T19:51:42.031632+0000', 'last_undegraded': '2024-04-26T19:51:42.031632+0000', 'last_fullsized': '2024-04-26T19:51:42.031632+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'last_clean_scrub_stamp': '2024-04-26T19:51:42.019054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-26T19:51:47.701 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T19:51:47.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.810+0000 7f43636c8640 1 Processor -- start 2024-04-26T19:51:47.811 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.810+0000 7f43636c8640 1 -- start start 2024-04-26T19:51:47.812 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f43636c8640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f435c0b4b90 0x7f435c06fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f43636c8640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f435c0c7a40 con 0x7f435c0d1430 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f43636c8640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f435c06f0a0 con 0x7f435c0c7460 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f43636c8640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f435c0700c0 con 0x7f435c0b4b90 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f4361464640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f435c0b4b90 0x7f435c06fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f4360c63640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f435c0d1430 0x7f435c0c6d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53620/0 (socket says 172.21.15.79:53620) 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f4361464640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f435c0b4b90 0x7f435c06fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49010/0 (socket says 172.21.15.79:49010) 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f4360c63640 1 -- 172.21.15.79:0/1235027645 learned_addr learned my addr 172.21.15.79:0/1235027645 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f4361464640 1 -- 172.21.15.79:0/1235027645 >> v1:172.21.15.83:6789/0 conn(0x7f435c0c7460 legacy=0x7f435c0c51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f4361464640 1 -- 172.21.15.79:0/1235027645 >> v1:172.21.15.79:6789/0 conn(0x7f435c0d1430 legacy=0x7f435c0c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f4361464640 1 -- 172.21.15.79:0/1235027645 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c06f380 con 0x7f435c0b4b90 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.811+0000 7f4353fff640 1 -- 172.21.15.79:0/1235027645 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1342722560 0 0) 0x7f435c0c7a40 con 0x7f435c0d1430 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.812+0000 7f4361464640 1 --2- 172.21.15.79:0/1235027645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 0x7f435c06fb80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f435800bfa0 tx=0x7f435802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6709f4efe13c218 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:47.813 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.812+0000 7f4353fff640 1 -- 172.21.15.79:0/1235027645 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358039070 con 0x7f435c0b4b90 2024-04-26T19:51:47.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.812+0000 7f4353fff640 1 -- 172.21.15.79:0/1235027645 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4358009540 con 0x7f435c0b4b90 2024-04-26T19:51:47.814 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.812+0000 7f4353fff640 1 -- 172.21.15.79:0/1235027645 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358009910 con 0x7f435c0b4b90 2024-04-26T19:51:47.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.813+0000 7f43636c8640 1 -- 172.21.15.79:0/1235027645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 msgr2=0x7f435c06fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.813+0000 7f43636c8640 1 --2- 172.21.15.79:0/1235027645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 0x7f435c06fb80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f435800bfa0 tx=0x7f435802c8b0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.813+0000 7f43636c8640 1 -- 172.21.15.79:0/1235027645 shutdown_connections 2024-04-26T19:51:47.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.813+0000 7f43636c8640 1 --2- 172.21.15.79:0/1235027645 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 0x7f435c06fb80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.813+0000 7f43636c8640 1 -- 172.21.15.79:0/1235027645 >> 172.21.15.79:0/1235027645 conn(0x7f435c073820 msgr2=0x7f435c0d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:47.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.813+0000 7f43636c8640 1 -- 172.21.15.79:0/1235027645 shutdown_connections 2024-04-26T19:51:47.817 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.813+0000 7f43636c8640 1 -- 172.21.15.79:0/1235027645 wait complete. 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.814+0000 7f43636c8640 1 Processor -- start 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.814+0000 7f43636c8640 1 -- start start 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f43636c8640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 0x7f435c072870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f43636c8640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f435c0c7460 0x7f435c146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f43636c8640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c0d1430 0x7f435c1e6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f43636c8640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f435c0b4f70 con 0x7f435c0b4b90 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f43636c8640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f435c0c7d20 con 0x7f435c0d1430 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f43636c8640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f435c0b50f0 con 0x7f435c0c7460 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4361464640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 0x7f435c072870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4361464640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 0x7f435c072870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49026/0 (socket says 172.21.15.79:49026) 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4361464640 1 -- 172.21.15.79:0/2761488501 learned_addr learned my addr 172.21.15.79:0/2761488501 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4361464640 1 -- 172.21.15.79:0/2761488501 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f435c0c7460 msgr2=0x7f435c146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.818 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4361464640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f435c0c7460 0x7f435c146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4361464640 1 -- 172.21.15.79:0/2761488501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c0d1430 msgr2=0x7f435c1e6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4361464640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c0d1430 0x7f435c1e6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4361464640 1 -- 172.21.15.79:0/2761488501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c1472a0 con 0x7f435c0b4b90 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4361464640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 0x7f435c072870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f435800bfa0 tx=0x7f4358004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4351ffb640 1 -- 172.21.15.79:0/2761488501 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358043020 con 0x7f435c0b4b90 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4351ffb640 1 -- 172.21.15.79:0/2761488501 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4358034660 con 0x7f435c0b4b90 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c0cfd40 con 0x7f435c0b4b90 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f4351ffb640 1 -- 172.21.15.79:0/2761488501 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4358039040 con 0x7f435c0b4b90 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.815+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c151ae0 con 0x7f435c0b4b90 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.816+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f435c0506c0 con 0x7f435c0b4b90 2024-04-26T19:51:47.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.817+0000 7f4351ffb640 1 -- 172.21.15.79:0/2761488501 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4358046020 con 0x7f435c0b4b90 2024-04-26T19:51:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.817+0000 7f4351ffb640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f434006c8f0 0x7f434006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.817+0000 7f4351ffb640 1 -- 172.21.15.79:0/2761488501 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f43580acc20 con 0x7f435c0b4b90 2024-04-26T19:51:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.817+0000 7f4360c63640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f434006c8f0 0x7f434006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:47.820 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.818+0000 7f4360c63640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f434006c8f0 0x7f434006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f434c007fd0 tx=0x7f434c007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:47.822 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:47.822+0000 7f4351ffb640 1 -- 172.21.15.79:0/2761488501 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f435804d030 con 0x7f435c0b4b90 2024-04-26T19:51:48.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.064+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f435c15d040 con 0x7f435c0b4b90 2024-04-26T19:51:48.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.065+0000 7f4351ffb640 1 -- 172.21.15.79:0/2761488501 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1270 (secure 0 0 0) 0x7f435c1472a0 con 0x7f435c0b4b90 2024-04-26T19:51:48.066 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:48.066 INFO:teuthology.orchestra.run.smithi079.stdout:{"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":101,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1714161017,"num_in_osds":8,"osd_in_since":1714161008,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225804288,"bytes_avail":772868308992,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5343}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-26T19:51:26.268132+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T19:51:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f434006c8f0 msgr2=0x7f434006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f434006c8f0 0x7f434006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f434c007fd0 tx=0x7f434c007920 comp rx=0 tx=0).stop 2024-04-26T19:51:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 msgr2=0x7f435c072870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 0x7f435c072870 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f435800bfa0 tx=0x7f4358004a40 comp rx=0 tx=0).stop 2024-04-26T19:51:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 shutdown_connections 2024-04-26T19:51:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f434006c8f0 0x7f434006edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f435c0d1430 0x7f435c1e6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.068 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f435c0c7460 0x7f435c146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 --2- 172.21.15.79:0/2761488501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f435c0b4b90 0x7f435c072870 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 >> 172.21.15.79:0/2761488501 conn(0x7f435c073820 msgr2=0x7f435c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 shutdown_connections 2024-04-26T19:51:48.069 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.067+0000 7f43636c8640 1 -- 172.21.15.79:0/2761488501 wait complete. 2024-04-26T19:51:48.079 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-26T19:51:48.079 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:51:48.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.175+0000 7fc5257da640 1 Processor -- start 2024-04-26T19:51:48.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.175+0000 7fc5257da640 1 -- start start 2024-04-26T19:51:48.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc5257da640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc520160390 0x7fc5200b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:48.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc5257da640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160770 0x7fc5200b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:48.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc5257da640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc5201630d0 0x7fc520163580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:48.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc5257da640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fc520144ca0 con 0x7fc5201630d0 2024-04-26T19:51:48.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc5257da640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc520163e00 con 0x7fc520160390 2024-04-26T19:51:48.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc5257da640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc520144f30 con 0x7fc520160770 2024-04-26T19:51:48.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51f7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc5201630d0 0x7fc520163580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:48.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51f7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc5201630d0 0x7fc520163580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49040/0 (socket says 172.21.15.79:49040) 2024-04-26T19:51:48.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51f7fe640 1 -- 172.21.15.79:0/1395129028 learned_addr learned my addr 172.21.15.79:0/1395129028 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:48.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51effd640 1 --2- 172.21.15.79:0/1395129028 >> v2:172.21.15.83:3300/0 conn(0x7fc520160390 0x7fc5200b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:48.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51e7fc640 1 --2- 172.21.15.79:0/1395129028 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160770 0x7fc5200b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:48.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51f7fe640 1 -- 172.21.15.79:0/1395129028 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160770 msgr2=0x7fc5200b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:48.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51f7fe640 1 --2- 172.21.15.79:0/1395129028 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160770 0x7fc5200b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51f7fe640 1 -- 172.21.15.79:0/1395129028 >> v2:172.21.15.83:3300/0 conn(0x7fc520160390 msgr2=0x7fc5200b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:48.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51f7fe640 1 --2- 172.21.15.79:0/1395129028 >> v2:172.21.15.83:3300/0 conn(0x7fc520160390 0x7fc5200b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.176+0000 7fc51f7fe640 1 -- 172.21.15.79:0/1395129028 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc520145210 con 0x7fc5201630d0 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.177+0000 7fc51f7fe640 1 --2- 172.21.15.79:0/1395129028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5201630d0 0x7fc520163580 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc5200b5690 tx=0x7fc51802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d56d8904c28591 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.177+0000 7fc51dffb640 1 -- 172.21.15.79:0/1395129028 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518037070 con 0x7fc5201630d0 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.177+0000 7fc51dffb640 1 -- 172.21.15.79:0/1395129028 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc518032440 con 0x7fc5201630d0 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.177+0000 7fc51dffb640 1 -- 172.21.15.79:0/1395129028 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518032810 con 0x7fc5201630d0 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.178+0000 7fc5257da640 1 -- 172.21.15.79:0/1395129028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5201630d0 msgr2=0x7fc520163580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.178+0000 7fc5257da640 1 --2- 172.21.15.79:0/1395129028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5201630d0 0x7fc520163580 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc5200b5690 tx=0x7fc51802a140 comp rx=0 tx=0).stop 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.178+0000 7fc5257da640 1 -- 172.21.15.79:0/1395129028 shutdown_connections 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.178+0000 7fc5257da640 1 --2- 172.21.15.79:0/1395129028 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5201630d0 0x7fc520163580 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.178+0000 7fc5257da640 1 --2- 172.21.15.79:0/1395129028 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160770 0x7fc5200b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.178+0000 7fc5257da640 1 --2- 172.21.15.79:0/1395129028 >> v2:172.21.15.83:3300/0 conn(0x7fc520160390 0x7fc5200b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.178+0000 7fc5257da640 1 -- 172.21.15.79:0/1395129028 >> 172.21.15.79:0/1395129028 conn(0x7fc5200b2720 msgr2=0x7fc520070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:48.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.178+0000 7fc5257da640 1 -- 172.21.15.79:0/1395129028 shutdown_connections 2024-04-26T19:51:48.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.178+0000 7fc5257da640 1 -- 172.21.15.79:0/1395129028 wait complete. 2024-04-26T19:51:48.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.179+0000 7fc5257da640 1 Processor -- start 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.179+0000 7fc5257da640 1 -- start start 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc5257da640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160390 0x7fc52015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc5257da640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520160770 0x7fc5201416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc5257da640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5201630d0 0x7fc5201f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc5257da640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc520145a10 con 0x7fc520160770 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc5257da640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc520163ac0 con 0x7fc5201630d0 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc5257da640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc520145b90 con 0x7fc520160390 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520160770 0x7fc5201416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520160770 0x7fc5201416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49048/0 (socket says 172.21.15.79:49048) 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51e7fc640 1 -- 172.21.15.79:0/272964551 learned_addr learned my addr 172.21.15.79:0/272964551 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:48.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51effd640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160390 0x7fc52015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51f7fe640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5201630d0 0x7fc5201f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51e7fc640 1 -- 172.21.15.79:0/272964551 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160390 msgr2=0x7fc52015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51e7fc640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160390 0x7fc52015b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51e7fc640 1 -- 172.21.15.79:0/272964551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5201630d0 msgr2=0x7fc5201f8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51e7fc640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5201630d0 0x7fc5201f8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51e7fc640 1 -- 172.21.15.79:0/272964551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5201f9790 con 0x7fc520160770 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.180+0000 7fc51e7fc640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520160770 0x7fc5201416a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc51400d5f0 tx=0x7fc51400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.181+0000 7fc5077fe640 1 -- 172.21.15.79:0/272964551 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5140125f0 con 0x7fc520160770 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.181+0000 7fc5077fe640 1 -- 172.21.15.79:0/272964551 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc514019070 con 0x7fc520160770 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.181+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5201f9a70 con 0x7fc520160770 2024-04-26T19:51:48.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.181+0000 7fc5077fe640 1 -- 172.21.15.79:0/272964551 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5140129d0 con 0x7fc520160770 2024-04-26T19:51:48.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.181+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc520141be0 con 0x7fc520160770 2024-04-26T19:51:48.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.182+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5200af980 con 0x7fc520160770 2024-04-26T19:51:48.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.182+0000 7fc5077fe640 1 -- 172.21.15.79:0/272964551 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc514012b70 con 0x7fc520160770 2024-04-26T19:51:48.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.182+0000 7fc5077fe640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc50c06c8f0 0x7fc50c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:48.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.182+0000 7fc51effd640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc50c06c8f0 0x7fc50c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:48.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.182+0000 7fc5077fe640 1 -- 172.21.15.79:0/272964551 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc51408c130 con 0x7fc520160770 2024-04-26T19:51:48.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.183+0000 7fc51effd640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc50c06c8f0 0x7fc50c06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc510004450 tx=0x7fc510003a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:48.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.187+0000 7fc5077fe640 1 -- 172.21.15.79:0/272964551 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc51405a3d0 con 0x7fc520160770 2024-04-26T19:51:48.361 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.361+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5200506c0 con 0x7fc50c06c8f0 2024-04-26T19:51:48.367 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.367+0000 7fc5077fe640 1 -- 172.21.15.79:0/272964551 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7fc5200506c0 con 0x7fc50c06c8f0 2024-04-26T19:51:48.370 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:48.371 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc50c06c8f0 msgr2=0x7fc50c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc50c06c8f0 0x7fc50c06edb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc510004450 tx=0x7fc510003a30 comp rx=0 tx=0).stop 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520160770 msgr2=0x7fc5201416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520160770 0x7fc5201416a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc51400d5f0 tx=0x7fc51400dab0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 shutdown_connections 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fc50c06c8f0 0x7fc50c06edb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc5201630d0 0x7fc5201f8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc520160770 0x7fc5201416a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 --2- 172.21.15.79:0/272964551 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc520160390 0x7fc52015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 >> 172.21.15.79:0/272964551 conn(0x7fc5200b2720 msgr2=0x7fc5200b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 shutdown_connections 2024-04-26T19:51:48.375 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:48.370+0000 7fc5257da640 1 -- 172.21.15.79:0/272964551 wait complete. 2024-04-26T19:51:48.381 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":78,"stamp":"2024-04-26T19:51:48.271824+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220908,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753812,"statfs":{"total":773094113280,"available":772867903488,"internally_reserved":0,"allocated":4820992,"data_stored":2742736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000996"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050068+0000","last_change":"2024-04-26T19:51:44.057653+0000","last_active":"2024-04-26T19:51:45.050068+0000","last_peered":"2024-04-26T19:51:45.050068+0000","last_clean":"2024-04-26T19:51:45.050068+0000","last_became_active":"2024-04-26T19:51:44.057379+0000","last_became_peered":"2024-04-26T19:51:44.057379+0000","last_unstale":"2024-04-26T19:51:45.050068+0000","last_undegraded":"2024-04-26T19:51:45.050068+0000","last_fullsized":"2024-04-26T19:51:45.050068+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:51.773418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049871+0000","last_change":"2024-04-26T19:51:44.052950+0000","last_active":"2024-04-26T19:51:45.049871+0000","last_peered":"2024-04-26T19:51:45.049871+0000","last_clean":"2024-04-26T19:51:45.049871+0000","last_became_active":"2024-04-26T19:51:44.052670+0000","last_became_peered":"2024-04-26T19:51:44.052670+0000","last_unstale":"2024-04-26T19:51:45.049871+0000","last_undegraded":"2024-04-26T19:51:45.049871+0000","last_fullsized":"2024-04-26T19:51:45.049871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:48.774437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060529+0000","last_change":"2024-04-26T19:51:44.056315+0000","last_active":"2024-04-26T19:51:44.060529+0000","last_peered":"2024-04-26T19:51:44.060529+0000","last_clean":"2024-04-26T19:51:44.060529+0000","last_became_active":"2024-04-26T19:51:44.056068+0000","last_became_peered":"2024-04-26T19:51:44.056068+0000","last_unstale":"2024-04-26T19:51:44.060529+0000","last_undegraded":"2024-04-26T19:51:44.060529+0000","last_fullsized":"2024-04-26T19:51:44.060529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:37.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645804+0000","last_change":"2024-04-26T19:51:44.051583+0000","last_active":"2024-04-26T19:51:45.645804+0000","last_peered":"2024-04-26T19:51:45.645804+0000","last_clean":"2024-04-26T19:51:45.645804+0000","last_became_active":"2024-04-26T19:51:44.051303+0000","last_became_peered":"2024-04-26T19:51:44.051303+0000","last_unstale":"2024-04-26T19:51:45.645804+0000","last_undegraded":"2024-04-26T19:51:45.645804+0000","last_fullsized":"2024-04-26T19:51:45.645804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:26.452557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645631+0000","last_change":"2024-04-26T19:51:44.047449+0000","last_active":"2024-04-26T19:51:45.645631+0000","last_peered":"2024-04-26T19:51:45.645631+0000","last_clean":"2024-04-26T19:51:45.645631+0000","last_became_active":"2024-04-26T19:51:44.047217+0000","last_became_peered":"2024-04-26T19:51:44.047217+0000","last_unstale":"2024-04-26T19:51:45.645631+0000","last_undegraded":"2024-04-26T19:51:45.645631+0000","last_fullsized":"2024-04-26T19:51:45.645631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:44.843944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.065591+0000","last_change":"2024-04-26T19:51:44.061255+0000","last_active":"2024-04-26T19:51:44.065591+0000","last_peered":"2024-04-26T19:51:44.065591+0000","last_clean":"2024-04-26T19:51:44.065591+0000","last_became_active":"2024-04-26T19:51:44.060988+0000","last_became_peered":"2024-04-26T19:51:44.060988+0000","last_unstale":"2024-04-26T19:51:44.065591+0000","last_undegraded":"2024-04-26T19:51:44.065591+0000","last_fullsized":"2024-04-26T19:51:44.065591+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:05:02.150722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038496+0000","last_change":"2024-04-26T19:51:44.054384+0000","last_active":"2024-04-26T19:51:46.038496+0000","last_peered":"2024-04-26T19:51:46.038496+0000","last_clean":"2024-04-26T19:51:46.038496+0000","last_became_active":"2024-04-26T19:51:44.054112+0000","last_became_peered":"2024-04-26T19:51:44.054112+0000","last_unstale":"2024-04-26T19:51:46.038496+0000","last_undegraded":"2024-04-26T19:51:46.038496+0000","last_fullsized":"2024-04-26T19:51:46.038496+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:28.264211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049759+0000","last_change":"2024-04-26T19:51:44.052078+0000","last_active":"2024-04-26T19:51:45.049759+0000","last_peered":"2024-04-26T19:51:45.049759+0000","last_clean":"2024-04-26T19:51:45.049759+0000","last_became_active":"2024-04-26T19:51:44.051840+0000","last_became_peered":"2024-04-26T19:51:44.051840+0000","last_unstale":"2024-04-26T19:51:45.049759+0000","last_undegraded":"2024-04-26T19:51:45.049759+0000","last_fullsized":"2024-04-26T19:51:45.049759+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:12:24.342993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060559+0000","last_change":"2024-04-26T19:51:44.055643+0000","last_active":"2024-04-26T19:51:44.060559+0000","last_peered":"2024-04-26T19:51:44.060559+0000","last_clean":"2024-04-26T19:51:44.060559+0000","last_became_active":"2024-04-26T19:51:44.055380+0000","last_became_peered":"2024-04-26T19:51:44.055380+0000","last_unstale":"2024-04-26T19:51:44.060559+0000","last_undegraded":"2024-04-26T19:51:44.060559+0000","last_fullsized":"2024-04-26T19:51:44.060559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:35.459759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038815+0000","last_change":"2024-04-26T19:51:44.055916+0000","last_active":"2024-04-26T19:51:46.038815+0000","last_peered":"2024-04-26T19:51:46.038815+0000","last_clean":"2024-04-26T19:51:46.038815+0000","last_became_active":"2024-04-26T19:51:44.055080+0000","last_became_peered":"2024-04-26T19:51:44.055080+0000","last_unstale":"2024-04-26T19:51:46.038815+0000","last_undegraded":"2024-04-26T19:51:46.038815+0000","last_fullsized":"2024-04-26T19:51:46.038815+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:26:28.839416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038866+0000","last_change":"2024-04-26T19:51:44.058213+0000","last_active":"2024-04-26T19:51:46.038866+0000","last_peered":"2024-04-26T19:51:46.038866+0000","last_clean":"2024-04-26T19:51:46.038866+0000","last_became_active":"2024-04-26T19:51:44.057958+0000","last_became_peered":"2024-04-26T19:51:44.057958+0000","last_unstale":"2024-04-26T19:51:46.038866+0000","last_undegraded":"2024-04-26T19:51:46.038866+0000","last_fullsized":"2024-04-26T19:51:46.038866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:49.952445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057282+0000","last_change":"2024-04-26T19:51:44.051551+0000","last_active":"2024-04-26T19:51:44.057282+0000","last_peered":"2024-04-26T19:51:44.057282+0000","last_clean":"2024-04-26T19:51:44.057282+0000","last_became_active":"2024-04-26T19:51:44.051280+0000","last_became_peered":"2024-04-26T19:51:44.051280+0000","last_unstale":"2024-04-26T19:51:44.057282+0000","last_undegraded":"2024-04-26T19:51:44.057282+0000","last_fullsized":"2024-04-26T19:51:44.057282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:04.576196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.039143+0000","last_change":"2024-04-26T19:51:44.056142+0000","last_active":"2024-04-26T19:51:46.039143+0000","last_peered":"2024-04-26T19:51:46.039143+0000","last_clean":"2024-04-26T19:51:46.039143+0000","last_became_active":"2024-04-26T19:51:44.055535+0000","last_became_peered":"2024-04-26T19:51:44.055535+0000","last_unstale":"2024-04-26T19:51:46.039143+0000","last_undegraded":"2024-04-26T19:51:46.039143+0000","last_fullsized":"2024-04-26T19:51:46.039143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:51.773418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.048874+0000","last_change":"2024-04-26T19:51:44.048345+0000","last_active":"2024-04-26T19:51:45.048874+0000","last_peered":"2024-04-26T19:51:45.048874+0000","last_clean":"2024-04-26T19:51:45.048874+0000","last_became_active":"2024-04-26T19:51:44.048065+0000","last_became_peered":"2024-04-26T19:51:44.048065+0000","last_unstale":"2024-04-26T19:51:45.048874+0000","last_undegraded":"2024-04-26T19:51:45.048874+0000","last_fullsized":"2024-04-26T19:51:45.048874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:04.576196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038562+0000","last_change":"2024-04-26T19:51:44.056391+0000","last_active":"2024-04-26T19:51:46.038562+0000","last_peered":"2024-04-26T19:51:46.038562+0000","last_clean":"2024-04-26T19:51:46.038562+0000","last_became_active":"2024-04-26T19:51:44.054742+0000","last_became_peered":"2024-04-26T19:51:44.054742+0000","last_unstale":"2024-04-26T19:51:46.038562+0000","last_undegraded":"2024-04-26T19:51:46.038562+0000","last_fullsized":"2024-04-26T19:51:46.038562+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:42.866288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182149+0000","last_change":"2024-04-26T19:51:44.055994+0000","last_active":"2024-04-26T19:51:46.182149+0000","last_peered":"2024-04-26T19:51:46.182149+0000","last_clean":"2024-04-26T19:51:46.182149+0000","last_became_active":"2024-04-26T19:51:44.055734+0000","last_became_peered":"2024-04-26T19:51:44.055734+0000","last_unstale":"2024-04-26T19:51:46.182149+0000","last_undegraded":"2024-04-26T19:51:46.182149+0000","last_fullsized":"2024-04-26T19:51:46.182149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:37.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182713+0000","last_change":"2024-04-26T19:51:44.055909+0000","last_active":"2024-04-26T19:51:46.182713+0000","last_peered":"2024-04-26T19:51:46.182713+0000","last_clean":"2024-04-26T19:51:46.182713+0000","last_became_active":"2024-04-26T19:51:44.055565+0000","last_became_peered":"2024-04-26T19:51:44.055565+0000","last_unstale":"2024-04-26T19:51:46.182713+0000","last_undegraded":"2024-04-26T19:51:46.182713+0000","last_fullsized":"2024-04-26T19:51:46.182713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:42.866288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057336+0000","last_change":"2024-04-26T19:51:44.048623+0000","last_active":"2024-04-26T19:51:44.057336+0000","last_peered":"2024-04-26T19:51:44.057336+0000","last_clean":"2024-04-26T19:51:44.057336+0000","last_became_active":"2024-04-26T19:51:44.048361+0000","last_became_peered":"2024-04-26T19:51:44.048361+0000","last_unstale":"2024-04-26T19:51:44.057336+0000","last_undegraded":"2024-04-26T19:51:44.057336+0000","last_fullsized":"2024-04-26T19:51:44.057336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:44.843944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645798+0000","last_change":"2024-04-26T19:51:44.048683+0000","last_active":"2024-04-26T19:51:45.645798+0000","last_peered":"2024-04-26T19:51:45.645798+0000","last_clean":"2024-04-26T19:51:45.645798+0000","last_became_active":"2024-04-26T19:51:44.048392+0000","last_became_peered":"2024-04-26T19:51:44.048392+0000","last_unstale":"2024-04-26T19:51:45.645798+0000","last_undegraded":"2024-04-26T19:51:45.645798+0000","last_fullsized":"2024-04-26T19:51:45.645798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:42.436894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.058371+0000","last_change":"2024-04-26T19:51:44.049275+0000","last_active":"2024-04-26T19:51:44.058371+0000","last_peered":"2024-04-26T19:51:44.058371+0000","last_clean":"2024-04-26T19:51:44.058371+0000","last_became_active":"2024-04-26T19:51:44.048962+0000","last_became_peered":"2024-04-26T19:51:44.048962+0000","last_unstale":"2024-04-26T19:51:44.058371+0000","last_undegraded":"2024-04-26T19:51:44.058371+0000","last_fullsized":"2024-04-26T19:51:44.058371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:42.436894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038601+0000","last_change":"2024-04-26T19:51:44.054992+0000","last_active":"2024-04-26T19:51:46.038601+0000","last_peered":"2024-04-26T19:51:46.038601+0000","last_clean":"2024-04-26T19:51:46.038601+0000","last_became_active":"2024-04-26T19:51:44.054217+0000","last_became_peered":"2024-04-26T19:51:44.054217+0000","last_unstale":"2024-04-26T19:51:46.038601+0000","last_undegraded":"2024-04-26T19:51:46.038601+0000","last_fullsized":"2024-04-26T19:51:46.038601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:51.764577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049324+0000","last_change":"2024-04-26T19:51:44.044045+0000","last_active":"2024-04-26T19:51:45.049324+0000","last_peered":"2024-04-26T19:51:45.049324+0000","last_clean":"2024-04-26T19:51:45.049324+0000","last_became_active":"2024-04-26T19:51:44.043745+0000","last_became_peered":"2024-04-26T19:51:44.043745+0000","last_unstale":"2024-04-26T19:51:45.049324+0000","last_undegraded":"2024-04-26T19:51:45.049324+0000","last_fullsized":"2024-04-26T19:51:45.049324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:17.443111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182157+0000","last_change":"2024-04-26T19:51:44.051746+0000","last_active":"2024-04-26T19:51:46.182157+0000","last_peered":"2024-04-26T19:51:46.182157+0000","last_clean":"2024-04-26T19:51:46.182157+0000","last_became_active":"2024-04-26T19:51:44.051444+0000","last_became_peered":"2024-04-26T19:51:44.051444+0000","last_unstale":"2024-04-26T19:51:46.182157+0000","last_undegraded":"2024-04-26T19:51:46.182157+0000","last_fullsized":"2024-04-26T19:51:46.182157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:46.511485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645598+0000","last_change":"2024-04-26T19:51:44.044956+0000","last_active":"2024-04-26T19:51:45.645598+0000","last_peered":"2024-04-26T19:51:45.645598+0000","last_clean":"2024-04-26T19:51:45.645598+0000","last_became_active":"2024-04-26T19:51:44.044694+0000","last_became_peered":"2024-04-26T19:51:44.044694+0000","last_unstale":"2024-04-26T19:51:45.645598+0000","last_undegraded":"2024-04-26T19:51:45.645598+0000","last_fullsized":"2024-04-26T19:51:45.645598+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:25.346009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057082+0000","last_change":"2024-04-26T19:51:44.042636+0000","last_active":"2024-04-26T19:51:44.057082+0000","last_peered":"2024-04-26T19:51:44.057082+0000","last_clean":"2024-04-26T19:51:44.057082+0000","last_became_active":"2024-04-26T19:51:44.042347+0000","last_became_peered":"2024-04-26T19:51:44.042347+0000","last_unstale":"2024-04-26T19:51:44.057082+0000","last_undegraded":"2024-04-26T19:51:44.057082+0000","last_fullsized":"2024-04-26T19:51:44.057082+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:25.346009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060552+0000","last_change":"2024-04-26T19:51:44.056686+0000","last_active":"2024-04-26T19:51:44.060552+0000","last_peered":"2024-04-26T19:51:44.060552+0000","last_clean":"2024-04-26T19:51:44.060552+0000","last_became_active":"2024-04-26T19:51:44.056412+0000","last_became_peered":"2024-04-26T19:51:44.056412+0000","last_unstale":"2024-04-26T19:51:44.060552+0000","last_undegraded":"2024-04-26T19:51:44.060552+0000","last_fullsized":"2024-04-26T19:51:44.060552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:48.006221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.046013+0000","last_change":"2024-04-26T19:51:44.038799+0000","last_active":"2024-04-26T19:51:44.046013+0000","last_peered":"2024-04-26T19:51:44.046013+0000","last_clean":"2024-04-26T19:51:44.046013+0000","last_became_active":"2024-04-26T19:51:44.038506+0000","last_became_peered":"2024-04-26T19:51:44.038506+0000","last_unstale":"2024-04-26T19:51:44.046013+0000","last_undegraded":"2024-04-26T19:51:44.046013+0000","last_fullsized":"2024-04-26T19:51:44.046013+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:29.958942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038848+0000","last_change":"2024-04-26T19:51:44.060010+0000","last_active":"2024-04-26T19:51:46.038848+0000","last_peered":"2024-04-26T19:51:46.038848+0000","last_clean":"2024-04-26T19:51:46.038848+0000","last_became_active":"2024-04-26T19:51:44.058543+0000","last_became_peered":"2024-04-26T19:51:44.058543+0000","last_unstale":"2024-04-26T19:51:46.038848+0000","last_undegraded":"2024-04-26T19:51:46.038848+0000","last_fullsized":"2024-04-26T19:51:46.038848+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:48.774437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.061361+0000","last_change":"2024-04-26T19:51:44.059230+0000","last_active":"2024-04-26T19:51:44.061361+0000","last_peered":"2024-04-26T19:51:44.061361+0000","last_clean":"2024-04-26T19:51:44.061361+0000","last_became_active":"2024-04-26T19:51:44.058955+0000","last_became_peered":"2024-04-26T19:51:44.058955+0000","last_unstale":"2024-04-26T19:51:44.061361+0000","last_undegraded":"2024-04-26T19:51:44.061361+0000","last_fullsized":"2024-04-26T19:51:44.061361+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:21.782979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.061395+0000","last_change":"2024-04-26T19:51:44.051651+0000","last_active":"2024-04-26T19:51:44.061395+0000","last_peered":"2024-04-26T19:51:44.061395+0000","last_clean":"2024-04-26T19:51:44.061395+0000","last_became_active":"2024-04-26T19:51:44.051379+0000","last_became_peered":"2024-04-26T19:51:44.051379+0000","last_unstale":"2024-04-26T19:51:44.061395+0000","last_undegraded":"2024-04-26T19:51:44.061395+0000","last_fullsized":"2024-04-26T19:51:44.061395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:26.452557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182493+0000","last_change":"2024-04-26T19:51:44.052620+0000","last_active":"2024-04-26T19:51:46.182493+0000","last_peered":"2024-04-26T19:51:46.182493+0000","last_clean":"2024-04-26T19:51:46.182493+0000","last_became_active":"2024-04-26T19:51:44.052131+0000","last_became_peered":"2024-04-26T19:51:44.052131+0000","last_unstale":"2024-04-26T19:51:46.182493+0000","last_undegraded":"2024-04-26T19:51:46.182493+0000","last_fullsized":"2024-04-26T19:51:46.182493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:24.558027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.056621+0000","last_change":"2024-04-26T19:51:44.052591+0000","last_active":"2024-04-26T19:51:44.056621+0000","last_peered":"2024-04-26T19:51:44.056621+0000","last_clean":"2024-04-26T19:51:44.056621+0000","last_became_active":"2024-04-26T19:51:44.052369+0000","last_became_peered":"2024-04-26T19:51:44.052369+0000","last_unstale":"2024-04-26T19:51:44.056621+0000","last_undegraded":"2024-04-26T19:51:44.056621+0000","last_fullsized":"2024-04-26T19:51:44.056621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:09:10.619763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182474+0000","last_change":"2024-04-26T19:51:44.056559+0000","last_active":"2024-04-26T19:51:46.182474+0000","last_peered":"2024-04-26T19:51:46.182474+0000","last_clean":"2024-04-26T19:51:46.182474+0000","last_became_active":"2024-04-26T19:51:44.056169+0000","last_became_peered":"2024-04-26T19:51:44.056169+0000","last_unstale":"2024-04-26T19:51:46.182474+0000","last_undegraded":"2024-04-26T19:51:46.182474+0000","last_fullsized":"2024-04-26T19:51:46.182474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:01.488308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.058285+0000","last_change":"2024-04-26T19:51:43.047367+0000","last_active":"2024-04-26T19:51:44.058285+0000","last_peered":"2024-04-26T19:51:44.058285+0000","last_clean":"2024-04-26T19:51:44.058285+0000","last_became_active":"2024-04-26T19:51:43.047075+0000","last_became_peered":"2024-04-26T19:51:43.047075+0000","last_unstale":"2024-04-26T19:51:44.058285+0000","last_undegraded":"2024-04-26T19:51:44.058285+0000","last_fullsized":"2024-04-26T19:51:44.058285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:07.628505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.065585+0000","last_change":"2024-04-26T19:51:44.061149+0000","last_active":"2024-04-26T19:51:44.065585+0000","last_peered":"2024-04-26T19:51:44.065585+0000","last_clean":"2024-04-26T19:51:44.065585+0000","last_became_active":"2024-04-26T19:51:44.060721+0000","last_became_peered":"2024-04-26T19:51:44.060721+0000","last_unstale":"2024-04-26T19:51:44.065585+0000","last_undegraded":"2024-04-26T19:51:44.065585+0000","last_fullsized":"2024-04-26T19:51:44.065585+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:37.856601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.036620+0000","last_change":"2024-04-26T19:51:43.048646+0000","last_active":"2024-04-26T19:51:44.036620+0000","last_peered":"2024-04-26T19:51:44.036620+0000","last_clean":"2024-04-26T19:51:44.036620+0000","last_became_active":"2024-04-26T19:51:43.048370+0000","last_became_peered":"2024-04-26T19:51:43.048370+0000","last_unstale":"2024-04-26T19:51:44.036620+0000","last_undegraded":"2024-04-26T19:51:44.036620+0000","last_fullsized":"2024-04-26T19:51:44.036620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:20.779063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049131+0000","last_change":"2024-04-26T19:51:44.048401+0000","last_active":"2024-04-26T19:51:45.049131+0000","last_peered":"2024-04-26T19:51:45.049131+0000","last_clean":"2024-04-26T19:51:45.049131+0000","last_became_active":"2024-04-26T19:51:44.048106+0000","last_became_peered":"2024-04-26T19:51:44.048106+0000","last_unstale":"2024-04-26T19:51:45.049131+0000","last_undegraded":"2024-04-26T19:51:45.049131+0000","last_fullsized":"2024-04-26T19:51:45.049131+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:46.511485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060843+0000","last_change":"2024-04-26T19:51:43.045945+0000","last_active":"2024-04-26T19:51:44.060843+0000","last_peered":"2024-04-26T19:51:44.060843+0000","last_clean":"2024-04-26T19:51:44.060843+0000","last_became_active":"2024-04-26T19:51:43.045693+0000","last_became_peered":"2024-04-26T19:51:43.045693+0000","last_unstale":"2024-04-26T19:51:44.060843+0000","last_undegraded":"2024-04-26T19:51:44.060843+0000","last_fullsized":"2024-04-26T19:51:44.060843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:20.779063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050078+0000","last_change":"2024-04-26T19:51:44.050436+0000","last_active":"2024-04-26T19:51:45.050078+0000","last_peered":"2024-04-26T19:51:45.050078+0000","last_clean":"2024-04-26T19:51:45.050078+0000","last_became_active":"2024-04-26T19:51:44.050188+0000","last_became_peered":"2024-04-26T19:51:44.050188+0000","last_unstale":"2024-04-26T19:51:45.050078+0000","last_undegraded":"2024-04-26T19:51:45.050078+0000","last_fullsized":"2024-04-26T19:51:45.050078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:15.189311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049071+0000","last_change":"2024-04-26T19:51:43.044349+0000","last_active":"2024-04-26T19:51:45.049071+0000","last_peered":"2024-04-26T19:51:45.049071+0000","last_clean":"2024-04-26T19:51:45.049071+0000","last_became_active":"2024-04-26T19:51:43.044048+0000","last_became_peered":"2024-04-26T19:51:43.044048+0000","last_unstale":"2024-04-26T19:51:45.049071+0000","last_undegraded":"2024-04-26T19:51:45.049071+0000","last_fullsized":"2024-04-26T19:51:45.049071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:41.432978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057167+0000","last_change":"2024-04-26T19:51:44.048846+0000","last_active":"2024-04-26T19:51:44.057167+0000","last_peered":"2024-04-26T19:51:44.057167+0000","last_clean":"2024-04-26T19:51:44.057167+0000","last_became_active":"2024-04-26T19:51:44.048593+0000","last_became_peered":"2024-04-26T19:51:44.048593+0000","last_unstale":"2024-04-26T19:51:44.057167+0000","last_undegraded":"2024-04-26T19:51:44.057167+0000","last_fullsized":"2024-04-26T19:51:44.057167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:37.856601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038684+0000","last_change":"2024-04-26T19:51:43.050469+0000","last_active":"2024-04-26T19:51:46.038684+0000","last_peered":"2024-04-26T19:51:46.038684+0000","last_clean":"2024-04-26T19:51:46.038684+0000","last_became_active":"2024-04-26T19:51:43.050200+0000","last_became_peered":"2024-04-26T19:51:43.050200+0000","last_unstale":"2024-04-26T19:51:46.038684+0000","last_undegraded":"2024-04-26T19:51:46.038684+0000","last_fullsized":"2024-04-26T19:51:46.038684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:09:09.615847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038770+0000","last_change":"2024-04-26T19:51:44.054330+0000","last_active":"2024-04-26T19:51:46.038770+0000","last_peered":"2024-04-26T19:51:46.038770+0000","last_clean":"2024-04-26T19:51:46.038770+0000","last_became_active":"2024-04-26T19:51:44.054031+0000","last_became_peered":"2024-04-26T19:51:44.054031+0000","last_unstale":"2024-04-26T19:51:46.038770+0000","last_undegraded":"2024-04-26T19:51:46.038770+0000","last_fullsized":"2024-04-26T19:51:46.038770+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:06:46.770763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182097+0000","last_change":"2024-04-26T19:51:43.047611+0000","last_active":"2024-04-26T19:51:46.182097+0000","last_peered":"2024-04-26T19:51:46.182097+0000","last_clean":"2024-04-26T19:51:46.182097+0000","last_became_active":"2024-04-26T19:51:43.047326+0000","last_became_peered":"2024-04-26T19:51:43.047326+0000","last_unstale":"2024-04-26T19:51:46.182097+0000","last_undegraded":"2024-04-26T19:51:46.182097+0000","last_fullsized":"2024-04-26T19:51:46.182097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:24.342092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049332+0000","last_change":"2024-04-26T19:51:44.059262+0000","last_active":"2024-04-26T19:51:45.049332+0000","last_peered":"2024-04-26T19:51:45.049332+0000","last_clean":"2024-04-26T19:51:45.049332+0000","last_became_active":"2024-04-26T19:51:44.058996+0000","last_became_peered":"2024-04-26T19:51:44.058996+0000","last_unstale":"2024-04-26T19:51:45.049332+0000","last_undegraded":"2024-04-26T19:51:45.049332+0000","last_fullsized":"2024-04-26T19:51:45.049332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:24.558027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.048058+0000","last_change":"2024-04-26T19:51:43.048540+0000","last_active":"2024-04-26T19:51:44.048058+0000","last_peered":"2024-04-26T19:51:44.048058+0000","last_clean":"2024-04-26T19:51:44.048058+0000","last_became_active":"2024-04-26T19:51:43.048278+0000","last_became_peered":"2024-04-26T19:51:43.048278+0000","last_unstale":"2024-04-26T19:51:44.048058+0000","last_undegraded":"2024-04-26T19:51:44.048058+0000","last_fullsized":"2024-04-26T19:51:44.048058+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:41.432978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050068+0000","last_change":"2024-04-26T19:51:44.051602+0000","last_active":"2024-04-26T19:51:45.050068+0000","last_peered":"2024-04-26T19:51:45.050068+0000","last_clean":"2024-04-26T19:51:45.050068+0000","last_became_active":"2024-04-26T19:51:44.051324+0000","last_became_peered":"2024-04-26T19:51:44.051324+0000","last_unstale":"2024-04-26T19:51:45.050068+0000","last_undegraded":"2024-04-26T19:51:45.050068+0000","last_fullsized":"2024-04-26T19:51:45.050068+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:31.097455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.048601+0000","last_change":"2024-04-26T19:51:43.049668+0000","last_active":"2024-04-26T19:51:45.048601+0000","last_peered":"2024-04-26T19:51:45.048601+0000","last_clean":"2024-04-26T19:51:45.048601+0000","last_became_active":"2024-04-26T19:51:43.049389+0000","last_became_peered":"2024-04-26T19:51:43.049389+0000","last_unstale":"2024-04-26T19:51:45.048601+0000","last_undegraded":"2024-04-26T19:51:45.048601+0000","last_fullsized":"2024-04-26T19:51:45.048601+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:43.840028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645839+0000","last_change":"2024-04-26T19:51:44.051971+0000","last_active":"2024-04-26T19:51:45.645839+0000","last_peered":"2024-04-26T19:51:45.645839+0000","last_clean":"2024-04-26T19:51:45.645839+0000","last_became_active":"2024-04-26T19:51:44.051694+0000","last_became_peered":"2024-04-26T19:51:44.051694+0000","last_unstale":"2024-04-26T19:51:45.645839+0000","last_undegraded":"2024-04-26T19:51:45.645839+0000","last_fullsized":"2024-04-26T19:51:45.645839+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:21.782979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050132+0000","last_change":"2024-04-26T19:51:43.046804+0000","last_active":"2024-04-26T19:51:45.050132+0000","last_peered":"2024-04-26T19:51:45.050132+0000","last_clean":"2024-04-26T19:51:45.050132+0000","last_became_active":"2024-04-26T19:51:43.046450+0000","last_became_peered":"2024-04-26T19:51:43.046450+0000","last_unstale":"2024-04-26T19:51:45.050132+0000","last_undegraded":"2024-04-26T19:51:45.050132+0000","last_fullsized":"2024-04-26T19:51:45.050132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:03.572279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182501+0000","last_change":"2024-04-26T19:51:44.061012+0000","last_active":"2024-04-26T19:51:46.182501+0000","last_peered":"2024-04-26T19:51:46.182501+0000","last_clean":"2024-04-26T19:51:46.182501+0000","last_became_active":"2024-04-26T19:51:44.060705+0000","last_became_peered":"2024-04-26T19:51:44.060705+0000","last_unstale":"2024-04-26T19:51:46.182501+0000","last_undegraded":"2024-04-26T19:51:46.182501+0000","last_fullsized":"2024-04-26T19:51:46.182501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:56.774722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038930+0000","last_change":"2024-04-26T19:51:43.045862+0000","last_active":"2024-04-26T19:51:46.038930+0000","last_peered":"2024-04-26T19:51:46.038930+0000","last_clean":"2024-04-26T19:51:46.038930+0000","last_became_active":"2024-04-26T19:51:43.045355+0000","last_became_peered":"2024-04-26T19:51:43.045355+0000","last_unstale":"2024-04-26T19:51:46.038930+0000","last_undegraded":"2024-04-26T19:51:46.038930+0000","last_fullsized":"2024-04-26T19:51:46.038930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:48.512804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057107+0000","last_change":"2024-04-26T19:51:44.044487+0000","last_active":"2024-04-26T19:51:44.057107+0000","last_peered":"2024-04-26T19:51:44.057107+0000","last_clean":"2024-04-26T19:51:44.057107+0000","last_became_active":"2024-04-26T19:51:44.044242+0000","last_became_peered":"2024-04-26T19:51:44.044242+0000","last_unstale":"2024-04-26T19:51:44.057107+0000","last_undegraded":"2024-04-26T19:51:44.057107+0000","last_fullsized":"2024-04-26T19:51:44.057107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:49.516721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038934+0000","last_change":"2024-04-26T19:51:43.045767+0000","last_active":"2024-04-26T19:51:46.038934+0000","last_peered":"2024-04-26T19:51:46.038934+0000","last_clean":"2024-04-26T19:51:46.038934+0000","last_became_active":"2024-04-26T19:51:43.045410+0000","last_became_peered":"2024-04-26T19:51:43.045410+0000","last_unstale":"2024-04-26T19:51:46.038934+0000","last_undegraded":"2024-04-26T19:51:46.038934+0000","last_fullsized":"2024-04-26T19:51:46.038934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:47.002305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049371+0000","last_change":"2024-04-26T19:51:44.045406+0000","last_active":"2024-04-26T19:51:45.049371+0000","last_peered":"2024-04-26T19:51:45.049371+0000","last_clean":"2024-04-26T19:51:45.049371+0000","last_became_active":"2024-04-26T19:51:44.045151+0000","last_became_peered":"2024-04-26T19:51:44.045151+0000","last_unstale":"2024-04-26T19:51:45.049371+0000","last_undegraded":"2024-04-26T19:51:45.049371+0000","last_fullsized":"2024-04-26T19:51:45.049371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:17.748061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050288+0000","last_change":"2024-04-26T19:51:43.049080+0000","last_active":"2024-04-26T19:51:45.050288+0000","last_peered":"2024-04-26T19:51:45.050288+0000","last_clean":"2024-04-26T19:51:45.050288+0000","last_became_active":"2024-04-26T19:51:43.048811+0000","last_became_peered":"2024-04-26T19:51:43.048811+0000","last_unstale":"2024-04-26T19:51:45.050288+0000","last_undegraded":"2024-04-26T19:51:45.050288+0000","last_fullsized":"2024-04-26T19:51:45.050288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:34.455843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057203+0000","last_change":"2024-04-26T19:51:44.049931+0000","last_active":"2024-04-26T19:51:44.057203+0000","last_peered":"2024-04-26T19:51:44.057203+0000","last_clean":"2024-04-26T19:51:44.057203+0000","last_became_active":"2024-04-26T19:51:44.049685+0000","last_became_peered":"2024-04-26T19:51:44.049685+0000","last_unstale":"2024-04-26T19:51:44.057203+0000","last_undegraded":"2024-04-26T19:51:44.057203+0000","last_fullsized":"2024-04-26T19:51:44.057203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:05:02.150722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050133+0000","last_change":"2024-04-26T19:51:43.050284+0000","last_active":"2024-04-26T19:51:45.050133+0000","last_peered":"2024-04-26T19:51:45.050133+0000","last_clean":"2024-04-26T19:51:45.050133+0000","last_became_active":"2024-04-26T19:51:43.049914+0000","last_became_peered":"2024-04-26T19:51:43.049914+0000","last_unstale":"2024-04-26T19:51:45.050133+0000","last_undegraded":"2024-04-26T19:51:45.050133+0000","last_fullsized":"2024-04-26T19:51:45.050133+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:36.468083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049770+0000","last_change":"2024-04-26T19:51:44.047575+0000","last_active":"2024-04-26T19:51:45.049770+0000","last_peered":"2024-04-26T19:51:45.049770+0000","last_clean":"2024-04-26T19:51:45.049770+0000","last_became_active":"2024-04-26T19:51:44.047306+0000","last_became_peered":"2024-04-26T19:51:44.047306+0000","last_unstale":"2024-04-26T19:51:45.049770+0000","last_undegraded":"2024-04-26T19:51:45.049770+0000","last_fullsized":"2024-04-26T19:51:45.049770+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:26:28.839416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.049689+0000","last_change":"2024-04-26T19:51:43.048681+0000","last_active":"2024-04-26T19:51:44.049689+0000","last_peered":"2024-04-26T19:51:44.049689+0000","last_clean":"2024-04-26T19:51:44.049689+0000","last_became_active":"2024-04-26T19:51:43.048395+0000","last_became_peered":"2024-04-26T19:51:43.048395+0000","last_unstale":"2024-04-26T19:51:44.049689+0000","last_undegraded":"2024-04-26T19:51:44.049689+0000","last_fullsized":"2024-04-26T19:51:44.049689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:18.241418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182339+0000","last_change":"2024-04-26T19:51:44.053335+0000","last_active":"2024-04-26T19:51:46.182339+0000","last_peered":"2024-04-26T19:51:46.182339+0000","last_clean":"2024-04-26T19:51:46.182339+0000","last_became_active":"2024-04-26T19:51:44.053064+0000","last_became_peered":"2024-04-26T19:51:44.053064+0000","last_unstale":"2024-04-26T19:51:46.182339+0000","last_undegraded":"2024-04-26T19:51:46.182339+0000","last_fullsized":"2024-04-26T19:51:46.182339+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:17.443111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.034754+0000","last_change":"2024-04-26T19:51:43.047181+0000","last_active":"2024-04-26T19:51:44.034754+0000","last_peered":"2024-04-26T19:51:44.034754+0000","last_clean":"2024-04-26T19:51:44.034754+0000","last_became_active":"2024-04-26T19:51:43.046880+0000","last_became_peered":"2024-04-26T19:51:43.046880+0000","last_unstale":"2024-04-26T19:51:44.034754+0000","last_undegraded":"2024-04-26T19:51:44.034754+0000","last_fullsized":"2024-04-26T19:51:44.034754+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:44.022728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050059+0000","last_change":"2024-04-26T19:51:44.058112+0000","last_active":"2024-04-26T19:51:45.050059+0000","last_peered":"2024-04-26T19:51:45.050059+0000","last_clean":"2024-04-26T19:51:45.050059+0000","last_became_active":"2024-04-26T19:51:44.057398+0000","last_became_peered":"2024-04-26T19:51:44.057398+0000","last_unstale":"2024-04-26T19:51:45.050059+0000","last_undegraded":"2024-04-26T19:51:45.050059+0000","last_fullsized":"2024-04-26T19:51:45.050059+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:15.775387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182066+0000","last_change":"2024-04-26T19:51:43.049832+0000","last_active":"2024-04-26T19:51:46.182066+0000","last_peered":"2024-04-26T19:51:46.182066+0000","last_clean":"2024-04-26T19:51:46.182066+0000","last_became_active":"2024-04-26T19:51:43.049559+0000","last_became_peered":"2024-04-26T19:51:43.049559+0000","last_unstale":"2024-04-26T19:51:46.182066+0000","last_undegraded":"2024-04-26T19:51:46.182066+0000","last_fullsized":"2024-04-26T19:51:46.182066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:28.955025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050262+0000","last_change":"2024-04-26T19:51:44.051037+0000","last_active":"2024-04-26T19:51:45.050262+0000","last_peered":"2024-04-26T19:51:45.050262+0000","last_clean":"2024-04-26T19:51:45.050262+0000","last_became_active":"2024-04-26T19:51:44.050782+0000","last_became_peered":"2024-04-26T19:51:44.050782+0000","last_unstale":"2024-04-26T19:51:45.050262+0000","last_undegraded":"2024-04-26T19:51:45.050262+0000","last_fullsized":"2024-04-26T19:51:45.050262+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:00:30.533407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.039023+0000","last_change":"2024-04-26T19:51:43.047262+0000","last_active":"2024-04-26T19:51:46.039023+0000","last_peered":"2024-04-26T19:51:46.039023+0000","last_clean":"2024-04-26T19:51:46.039023+0000","last_became_active":"2024-04-26T19:51:43.046905+0000","last_became_peered":"2024-04-26T19:51:43.046905+0000","last_unstale":"2024-04-26T19:51:46.039023+0000","last_undegraded":"2024-04-26T19:51:46.039023+0000","last_fullsized":"2024-04-26T19:51:46.039023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:36.852685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038931+0000","last_change":"2024-04-26T19:51:44.054753+0000","last_active":"2024-04-26T19:51:46.038931+0000","last_peered":"2024-04-26T19:51:46.038931+0000","last_clean":"2024-04-26T19:51:46.038931+0000","last_became_active":"2024-04-26T19:51:44.054353+0000","last_became_peered":"2024-04-26T19:51:44.054353+0000","last_unstale":"2024-04-26T19:51:46.038931+0000","last_undegraded":"2024-04-26T19:51:46.038931+0000","last_fullsized":"2024-04-26T19:51:46.038931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:15.775387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049136+0000","last_change":"2024-04-26T19:51:43.040875+0000","last_active":"2024-04-26T19:51:45.049136+0000","last_peered":"2024-04-26T19:51:45.049136+0000","last_clean":"2024-04-26T19:51:45.049136+0000","last_became_active":"2024-04-26T19:51:43.040590+0000","last_became_peered":"2024-04-26T19:51:43.040590+0000","last_unstale":"2024-04-26T19:51:45.049136+0000","last_undegraded":"2024-04-26T19:51:45.049136+0000","last_fullsized":"2024-04-26T19:51:45.049136+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:20.779063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049028+0000","last_change":"2024-04-26T19:51:44.050029+0000","last_active":"2024-04-26T19:51:45.049028+0000","last_peered":"2024-04-26T19:51:45.049028+0000","last_clean":"2024-04-26T19:51:45.049028+0000","last_became_active":"2024-04-26T19:51:44.049607+0000","last_became_peered":"2024-04-26T19:51:44.049607+0000","last_unstale":"2024-04-26T19:51:45.049028+0000","last_undegraded":"2024-04-26T19:51:45.049028+0000","last_fullsized":"2024-04-26T19:51:45.049028+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:37.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645695+0000","last_change":"2024-04-26T19:51:43.048029+0000","last_active":"2024-04-26T19:51:45.645695+0000","last_peered":"2024-04-26T19:51:45.645695+0000","last_clean":"2024-04-26T19:51:45.645695+0000","last_became_active":"2024-04-26T19:51:43.047727+0000","last_became_peered":"2024-04-26T19:51:43.047727+0000","last_unstale":"2024-04-26T19:51:45.645695+0000","last_undegraded":"2024-04-26T19:51:45.645695+0000","last_fullsized":"2024-04-26T19:51:45.645695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:28.110640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645903+0000","last_change":"2024-04-26T19:51:44.050733+0000","last_active":"2024-04-26T19:51:45.645903+0000","last_peered":"2024-04-26T19:51:45.645903+0000","last_clean":"2024-04-26T19:51:45.645903+0000","last_became_active":"2024-04-26T19:51:44.050379+0000","last_became_peered":"2024-04-26T19:51:44.050379+0000","last_unstale":"2024-04-26T19:51:45.645903+0000","last_undegraded":"2024-04-26T19:51:45.645903+0000","last_fullsized":"2024-04-26T19:51:45.645903+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:19.245335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.048172+0000","last_change":"2024-04-26T19:51:43.045943+0000","last_active":"2024-04-26T19:51:44.048172+0000","last_peered":"2024-04-26T19:51:44.048172+0000","last_clean":"2024-04-26T19:51:44.048172+0000","last_became_active":"2024-04-26T19:51:43.045562+0000","last_became_peered":"2024-04-26T19:51:43.045562+0000","last_unstale":"2024-04-26T19:51:44.048172+0000","last_undegraded":"2024-04-26T19:51:44.048172+0000","last_fullsized":"2024-04-26T19:51:44.048172+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:25.448641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182580+0000","last_change":"2024-04-26T19:51:44.046440+0000","last_active":"2024-04-26T19:51:46.182580+0000","last_peered":"2024-04-26T19:51:46.182580+0000","last_clean":"2024-04-26T19:51:46.182580+0000","last_became_active":"2024-04-26T19:51:44.045900+0000","last_became_peered":"2024-04-26T19:51:44.045900+0000","last_unstale":"2024-04-26T19:51:46.182580+0000","last_undegraded":"2024-04-26T19:51:46.182580+0000","last_fullsized":"2024-04-26T19:51:46.182580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:51.418205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645593+0000","last_change":"2024-04-26T19:51:43.050100+0000","last_active":"2024-04-26T19:51:45.645593+0000","last_peered":"2024-04-26T19:51:45.645593+0000","last_clean":"2024-04-26T19:51:45.645593+0000","last_became_active":"2024-04-26T19:51:43.049842+0000","last_became_peered":"2024-04-26T19:51:43.049842+0000","last_unstale":"2024-04-26T19:51:45.645593+0000","last_undegraded":"2024-04-26T19:51:45.645593+0000","last_fullsized":"2024-04-26T19:51:45.645593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:15.689198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057568+0000","last_change":"2024-04-26T19:51:44.046563+0000","last_active":"2024-04-26T19:51:44.057568+0000","last_peered":"2024-04-26T19:51:44.057568+0000","last_clean":"2024-04-26T19:51:44.057568+0000","last_became_active":"2024-04-26T19:51:44.046322+0000","last_became_peered":"2024-04-26T19:51:44.046322+0000","last_unstale":"2024-04-26T19:51:44.057568+0000","last_undegraded":"2024-04-26T19:51:44.057568+0000","last_fullsized":"2024-04-26T19:51:44.057568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:48.006221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645580+0000","last_change":"2024-04-26T19:51:43.042457+0000","last_active":"2024-04-26T19:51:45.645580+0000","last_peered":"2024-04-26T19:51:45.645580+0000","last_clean":"2024-04-26T19:51:45.645580+0000","last_became_active":"2024-04-26T19:51:43.042081+0000","last_became_peered":"2024-04-26T19:51:43.042081+0000","last_unstale":"2024-04-26T19:51:45.645580+0000","last_undegraded":"2024-04-26T19:51:45.645580+0000","last_fullsized":"2024-04-26T19:51:45.645580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:42:14.235749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038755+0000","last_change":"2024-04-26T19:51:44.054784+0000","last_active":"2024-04-26T19:51:46.038755+0000","last_peered":"2024-04-26T19:51:46.038755+0000","last_clean":"2024-04-26T19:51:46.038755+0000","last_became_active":"2024-04-26T19:51:44.054399+0000","last_became_peered":"2024-04-26T19:51:44.054399+0000","last_unstale":"2024-04-26T19:51:46.038755+0000","last_undegraded":"2024-04-26T19:51:46.038755+0000","last_fullsized":"2024-04-26T19:51:46.038755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:12:24.342993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182262+0000","last_change":"2024-04-26T19:51:43.049855+0000","last_active":"2024-04-26T19:51:46.182262+0000","last_peered":"2024-04-26T19:51:46.182262+0000","last_clean":"2024-04-26T19:51:46.182262+0000","last_became_active":"2024-04-26T19:51:43.049519+0000","last_became_peered":"2024-04-26T19:51:43.049519+0000","last_unstale":"2024-04-26T19:51:46.182262+0000","last_undegraded":"2024-04-26T19:51:46.182262+0000","last_fullsized":"2024-04-26T19:51:46.182262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:41.432978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050183+0000","last_change":"2024-04-26T19:51:44.052190+0000","last_active":"2024-04-26T19:51:45.050183+0000","last_peered":"2024-04-26T19:51:45.050183+0000","last_clean":"2024-04-26T19:51:45.050183+0000","last_became_active":"2024-04-26T19:51:44.051942+0000","last_became_peered":"2024-04-26T19:51:44.051942+0000","last_unstale":"2024-04-26T19:51:45.050183+0000","last_undegraded":"2024-04-26T19:51:45.050183+0000","last_fullsized":"2024-04-26T19:51:45.050183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:14.527062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645498+0000","last_change":"2024-04-26T19:51:43.044089+0000","last_active":"2024-04-26T19:51:45.645498+0000","last_peered":"2024-04-26T19:51:45.645498+0000","last_clean":"2024-04-26T19:51:45.645498+0000","last_became_active":"2024-04-26T19:51:43.043839+0000","last_became_peered":"2024-04-26T19:51:43.043839+0000","last_unstale":"2024-04-26T19:51:45.645498+0000","last_undegraded":"2024-04-26T19:51:45.645498+0000","last_fullsized":"2024-04-26T19:51:45.645498+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:26.064251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060362+0000","last_change":"2024-04-26T19:51:44.056608+0000","last_active":"2024-04-26T19:51:44.060362+0000","last_peered":"2024-04-26T19:51:44.060362+0000","last_clean":"2024-04-26T19:51:44.060362+0000","last_became_active":"2024-04-26T19:51:44.056324+0000","last_became_peered":"2024-04-26T19:51:44.056324+0000","last_unstale":"2024-04-26T19:51:44.060362+0000","last_undegraded":"2024-04-26T19:51:44.060362+0000","last_fullsized":"2024-04-26T19:51:44.060362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:49.516721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.039083+0000","last_change":"2024-04-26T19:51:43.046179+0000","last_active":"2024-04-26T19:51:46.039083+0000","last_peered":"2024-04-26T19:51:46.039083+0000","last_clean":"2024-04-26T19:51:46.039083+0000","last_became_active":"2024-04-26T19:51:43.045671+0000","last_became_peered":"2024-04-26T19:51:43.045671+0000","last_unstale":"2024-04-26T19:51:46.039083+0000","last_undegraded":"2024-04-26T19:51:46.039083+0000","last_fullsized":"2024-04-26T19:51:46.039083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:05:01.146806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182600+0000","last_change":"2024-04-26T19:51:44.061767+0000","last_active":"2024-04-26T19:51:46.182600+0000","last_peered":"2024-04-26T19:51:46.182600+0000","last_clean":"2024-04-26T19:51:46.182600+0000","last_became_active":"2024-04-26T19:51:44.061480+0000","last_became_peered":"2024-04-26T19:51:44.061480+0000","last_unstale":"2024-04-26T19:51:46.182600+0000","last_undegraded":"2024-04-26T19:51:46.182600+0000","last_fullsized":"2024-04-26T19:51:46.182600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:28.264211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645787+0000","last_change":"2024-04-26T19:51:43.040495+0000","last_active":"2024-04-26T19:51:45.645787+0000","last_peered":"2024-04-26T19:51:45.645787+0000","last_clean":"2024-04-26T19:51:45.645787+0000","last_became_active":"2024-04-26T19:51:43.040233+0000","last_became_peered":"2024-04-26T19:51:43.040233+0000","last_unstale":"2024-04-26T19:51:45.645787+0000","last_undegraded":"2024-04-26T19:51:45.645787+0000","last_fullsized":"2024-04-26T19:51:45.645787+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:55:08.794958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049931+0000","last_change":"2024-04-26T19:51:44.049182+0000","last_active":"2024-04-26T19:51:45.049931+0000","last_peered":"2024-04-26T19:51:45.049931+0000","last_clean":"2024-04-26T19:51:45.049931+0000","last_became_active":"2024-04-26T19:51:44.048926+0000","last_became_peered":"2024-04-26T19:51:44.048926+0000","last_unstale":"2024-04-26T19:51:45.049931+0000","last_undegraded":"2024-04-26T19:51:45.049931+0000","last_fullsized":"2024-04-26T19:51:45.049931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:49.952445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.036794+0000","last_change":"2024-04-26T19:51:43.050964+0000","last_active":"2024-04-26T19:51:44.036794+0000","last_peered":"2024-04-26T19:51:44.036794+0000","last_clean":"2024-04-26T19:51:44.036794+0000","last_became_active":"2024-04-26T19:51:43.050731+0000","last_became_peered":"2024-04-26T19:51:43.050731+0000","last_unstale":"2024-04-26T19:51:44.036794+0000","last_undegraded":"2024-04-26T19:51:44.036794+0000","last_fullsized":"2024-04-26T19:51:44.036794+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:09:09.615847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057223+0000","last_change":"2024-04-26T19:51:44.049391+0000","last_active":"2024-04-26T19:51:44.057223+0000","last_peered":"2024-04-26T19:51:44.057223+0000","last_clean":"2024-04-26T19:51:44.057223+0000","last_became_active":"2024-04-26T19:51:44.049156+0000","last_became_peered":"2024-04-26T19:51:44.049156+0000","last_unstale":"2024-04-26T19:51:44.057223+0000","last_undegraded":"2024-04-26T19:51:44.057223+0000","last_fullsized":"2024-04-26T19:51:44.057223+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:29.530604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.036620+0000","last_change":"2024-04-26T19:51:43.050051+0000","last_active":"2024-04-26T19:51:44.036620+0000","last_peered":"2024-04-26T19:51:44.036620+0000","last_clean":"2024-04-26T19:51:44.036620+0000","last_became_active":"2024-04-26T19:51:43.049770+0000","last_became_peered":"2024-04-26T19:51:43.049770+0000","last_unstale":"2024-04-26T19:51:44.036620+0000","last_undegraded":"2024-04-26T19:51:44.036620+0000","last_fullsized":"2024-04-26T19:51:44.036620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:18.241418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.065596+0000","last_change":"2024-04-26T19:51:44.061915+0000","last_active":"2024-04-26T19:51:44.065596+0000","last_peered":"2024-04-26T19:51:44.065596+0000","last_clean":"2024-04-26T19:51:44.065596+0000","last_became_active":"2024-04-26T19:51:44.061626+0000","last_became_peered":"2024-04-26T19:51:44.061626+0000","last_unstale":"2024-04-26T19:51:44.065596+0000","last_undegraded":"2024-04-26T19:51:44.065596+0000","last_fullsized":"2024-04-26T19:51:44.065596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:29.530604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.034556+0000","last_change":"2024-04-26T19:51:43.052188+0000","last_active":"2024-04-26T19:51:44.034556+0000","last_peered":"2024-04-26T19:51:44.034556+0000","last_clean":"2024-04-26T19:51:44.034556+0000","last_became_active":"2024-04-26T19:51:43.051996+0000","last_became_peered":"2024-04-26T19:51:43.051996+0000","last_unstale":"2024-04-26T19:51:44.034556+0000","last_undegraded":"2024-04-26T19:51:44.034556+0000","last_fullsized":"2024-04-26T19:51:44.034556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:19.093912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.048974+0000","last_change":"2024-04-26T19:51:44.049649+0000","last_active":"2024-04-26T19:51:45.048974+0000","last_peered":"2024-04-26T19:51:45.048974+0000","last_clean":"2024-04-26T19:51:45.048974+0000","last_became_active":"2024-04-26T19:51:44.049381+0000","last_became_peered":"2024-04-26T19:51:44.049381+0000","last_unstale":"2024-04-26T19:51:45.048974+0000","last_undegraded":"2024-04-26T19:51:45.048974+0000","last_fullsized":"2024-04-26T19:51:45.048974+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:35.459759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.048640+0000","last_change":"2024-04-26T19:51:43.045668+0000","last_active":"2024-04-26T19:51:45.048640+0000","last_peered":"2024-04-26T19:51:45.048640+0000","last_clean":"2024-04-26T19:51:45.048640+0000","last_became_active":"2024-04-26T19:51:43.045318+0000","last_became_peered":"2024-04-26T19:51:43.045318+0000","last_unstale":"2024-04-26T19:51:45.048640+0000","last_undegraded":"2024-04-26T19:51:45.048640+0000","last_fullsized":"2024-04-26T19:51:45.048640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:25.448641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182464+0000","last_change":"2024-04-26T19:51:44.060016+0000","last_active":"2024-04-26T19:51:46.182464+0000","last_peered":"2024-04-26T19:51:46.182464+0000","last_clean":"2024-04-26T19:51:46.182464+0000","last_became_active":"2024-04-26T19:51:44.059746+0000","last_became_peered":"2024-04-26T19:51:44.059746+0000","last_unstale":"2024-04-26T19:51:46.182464+0000","last_undegraded":"2024-04-26T19:51:46.182464+0000","last_fullsized":"2024-04-26T19:51:46.182464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:17.748061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049722+0000","last_change":"2024-04-26T19:51:43.045649+0000","last_active":"2024-04-26T19:51:45.049722+0000","last_peered":"2024-04-26T19:51:45.049722+0000","last_clean":"2024-04-26T19:51:45.049722+0000","last_became_active":"2024-04-26T19:51:43.045411+0000","last_became_peered":"2024-04-26T19:51:43.045411+0000","last_unstale":"2024-04-26T19:51:45.049722+0000","last_undegraded":"2024-04-26T19:51:45.049722+0000","last_fullsized":"2024-04-26T19:51:45.049722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:28.526687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038986+0000","last_change":"2024-04-26T19:51:44.058983+0000","last_active":"2024-04-26T19:51:46.038986+0000","last_peered":"2024-04-26T19:51:46.038986+0000","last_clean":"2024-04-26T19:51:46.038986+0000","last_became_active":"2024-04-26T19:51:44.058667+0000","last_became_peered":"2024-04-26T19:51:44.058667+0000","last_unstale":"2024-04-26T19:51:46.038986+0000","last_undegraded":"2024-04-26T19:51:46.038986+0000","last_fullsized":"2024-04-26T19:51:46.038986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:31.097455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182398+0000","last_change":"2024-04-26T19:51:43.048837+0000","last_active":"2024-04-26T19:51:46.182398+0000","last_peered":"2024-04-26T19:51:46.182398+0000","last_clean":"2024-04-26T19:51:46.182398+0000","last_became_active":"2024-04-26T19:51:43.048572+0000","last_became_peered":"2024-04-26T19:51:43.048572+0000","last_unstale":"2024-04-26T19:51:46.182398+0000","last_undegraded":"2024-04-26T19:51:46.182398+0000","last_fullsized":"2024-04-26T19:51:46.182398+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:43.840028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":205,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182412+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:51:46.182412+0000","last_peered":"2024-04-26T19:51:46.182412+0000","last_clean":"2024-04-26T19:51:46.182412+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:51:46.182412+0000","last_undegraded":"2024-04-26T19:51:46.182412+0000","last_fullsized":"2024-04-26T19:51:46.182412+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":789207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":191047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":197269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":197761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":198803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":197227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":190555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":780867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:51:48.802 INFO:tasks.ceph.mds.a.smithi079.stderr:2024-04-26T19:51:48.801+0000 7f62b1b83640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T19:51:51.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:51:51.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.494+0000 7f685694e640 1 Processor -- start 2024-04-26T19:51:51.495 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.494+0000 7f685694e640 1 -- start start 2024-04-26T19:51:51.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f685694e640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f6850156670 0x7f685015d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f685694e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68500cd9c0 0x7f68500b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f685694e640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f685015d670 con 0x7f68500735a0 2024-04-26T19:51:51.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f685694e640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f6850073110 con 0x7f6850156670 2024-04-26T19:51:51.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f685694e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f68500ce1a0 con 0x7f68500cd9c0 2024-04-26T19:51:51.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f6854eeb640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f68500735a0 0x7f685015fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53628/0 (socket says 172.21.15.79:53628) 2024-04-26T19:51:51.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f6854eeb640 1 -- 172.21.15.79:0/1084066545 learned_addr learned my addr 172.21.15.79:0/1084066545 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:51.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684f7fe640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68500cd9c0 0x7f68500b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4019599101 0 0) 0x7f6848001490 con 0x7f68500735a0 2024-04-26T19:51:51.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6838003520 con 0x7f68500735a0 2024-04-26T19:51:51.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684ffff640 1 --2- 172.21.15.79:0/1084066545 >> v2:172.21.15.83:3300/0 conn(0x7f6850156670 0x7f685015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 42368482 0 0) 0x7f6838003520 con 0x7f68500735a0 2024-04-26T19:51:51.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68500cd9c0 msgr2=0x7f68500b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:51.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684effd640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68500cd9c0 0x7f68500b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 >> v2:172.21.15.83:3300/0 conn(0x7f6850156670 msgr2=0x7f685015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:51.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684effd640 1 --2- 172.21.15.79:0/1084066545 >> v2:172.21.15.83:3300/0 conn(0x7f6850156670 0x7f685015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68500cdec0 con 0x7f68500735a0 2024-04-26T19:51:51.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.495+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f6848001bc0 con 0x7f68500735a0 2024-04-26T19:51:51.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 >> v1:172.21.15.79:6789/0 conn(0x7f68500735a0 legacy=0x7f685015fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:51.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684effd640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68500cd9c0 0x7f68380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684effd640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f6838004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684effd640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6838004d40 0x7f68380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6850073110 con 0x7f6838003f30 2024-04-26T19:51:51.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f68500ce1a0 con 0x7f6838004d40 2024-04-26T19:51:51.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f683800b9c0 con 0x7f68500cd9c0 2024-04-26T19:51:51.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684f7fe640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f6838004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684ffff640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68500cd9c0 0x7f68380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f6854eeb640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6838004d40 0x7f68380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684f7fe640 1 -- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68500cd9c0 msgr2=0x7f68380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:51.507 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684f7fe640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68500cd9c0 0x7f68380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684f7fe640 1 -- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6838004d40 msgr2=0x7f68380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684f7fe640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6838004d40 0x7f68380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.496+0000 7f684f7fe640 1 -- 172.21.15.79:0/1084066545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6838003520 con 0x7f6838003f30 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f684f7fe640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f6838004800 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6844009ac0 tx=0x7f6838005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581be179bc71fc3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844036070 con 0x7f6838003f30 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6844031400 con 0x7f6838003f30 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f684effd640 1 -- 172.21.15.79:0/1084066545 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6844009690 con 0x7f6838003f30 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f685694e640 1 -- 172.21.15.79:0/1084066545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 msgr2=0x7f6838004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f685694e640 1 --2- 172.21.15.79:0/1084066545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f6838004800 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6844009ac0 tx=0x7f6838005830 comp rx=0 tx=0).stop 2024-04-26T19:51:51.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f6854eeb640 1 -- 172.21.15.79:0/1084066545 reap_dead start 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f685694e640 1 -- 172.21.15.79:0/1084066545 shutdown_connections 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f685694e640 1 -- 172.21.15.79:0/1084066545 >> 172.21.15.79:0/1084066545 conn(0x7f68500b39d0 msgr2=0x7f685006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f685694e640 1 -- 172.21.15.79:0/1084066545 shutdown_connections 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.497+0000 7f685694e640 1 -- 172.21.15.79:0/1084066545 wait complete. 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.499+0000 7f685694e640 1 Processor -- start 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.499+0000 7f685694e640 1 -- start start 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.499+0000 7f685694e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f685006f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.499+0000 7f685694e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6850157840 0x7f6850156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.499+0000 7f685694e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68501482f0 0x7f6850071810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.499+0000 7f685694e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6850156a50 con 0x7f6838003f30 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.499+0000 7f685694e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6850072da0 con 0x7f6850157840 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.499+0000 7f685694e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f6850156bd0 con 0x7f68501482f0 2024-04-26T19:51:51.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f685006f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684ffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f685006f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49064/0 (socket says 172.21.15.79:49064) 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684ffff640 1 -- 172.21.15.79:0/1034978788 learned_addr learned my addr 172.21.15.79:0/1034978788 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684f7fe640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6850157840 0x7f6850156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f6854eeb640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68501482f0 0x7f6850071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684ffff640 1 -- 172.21.15.79:0/1034978788 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68501482f0 msgr2=0x7f6850071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684ffff640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68501482f0 0x7f6850071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684ffff640 1 -- 172.21.15.79:0/1034978788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6850157840 msgr2=0x7f6850156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684ffff640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6850157840 0x7f6850156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684ffff640 1 -- 172.21.15.79:0/1034978788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6850157cc0 con 0x7f6838003f30 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684ffff640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f685006f880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6840011a70 tx=0x7f6840011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:51.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684cff9640 1 -- 172.21.15.79:0/1034978788 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f684000fa90 con 0x7f6838003f30 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684cff9640 1 -- 172.21.15.79:0/1034978788 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f684000fc30 con 0x7f6838003f30 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685006eac0 con 0x7f6838003f30 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f684cff9640 1 -- 172.21.15.79:0/1034978788 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840014440 con 0x7f6838003f30 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.500+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685006f080 con 0x7f6838003f30 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.501+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68500506c0 con 0x7f6838003f30 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.502+0000 7f684cff9640 1 -- 172.21.15.79:0/1034978788 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6840012040 con 0x7f6838003f30 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.502+0000 7f684cff9640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f683c06c920 0x7f683c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.502+0000 7f684f7fe640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f683c06c920 0x7f683c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.502+0000 7f684cff9640 1 -- 172.21.15.79:0/1034978788 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f684008c280 con 0x7f6838003f30 2024-04-26T19:51:51.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.503+0000 7f684f7fe640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f683c06c920 0x7f683c06ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6844033000 tx=0x7f6844008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:51.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.507+0000 7f684cff9640 1 -- 172.21.15.79:0/1034978788 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6840019030 con 0x7f6838003f30 2024-04-26T19:51:51.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.677+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f685015e0e0 con 0x7f683c06c920 2024-04-26T19:51:51.683 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.682+0000 7f684cff9640 1 -- 172.21.15.79:0/1034978788 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7f685015e0e0 con 0x7f683c06c920 2024-04-26T19:51:51.686 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:51.688 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:51:51.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f683c06c920 msgr2=0x7f683c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:51.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f683c06c920 0x7f683c06ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6844033000 tx=0x7f6844008030 comp rx=0 tx=0).stop 2024-04-26T19:51:51.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 msgr2=0x7f685006f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:51.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f685006f880 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6840011a70 tx=0x7f6840011e70 comp rx=0 tx=0).stop 2024-04-26T19:51:51.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 shutdown_connections 2024-04-26T19:51:51.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f683c06c920 0x7f683c06ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f68501482f0 0x7f6850071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6850157840 0x7f6850156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 --2- 172.21.15.79:0/1034978788 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6838003f30 0x7f685006f880 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 >> 172.21.15.79:0/1034978788 conn(0x7f68500b39d0 msgr2=0x7f685015ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:51.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 shutdown_connections 2024-04-26T19:51:51.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.686+0000 7f685694e640 1 -- 172.21.15.79:0/1034978788 wait complete. 2024-04-26T19:51:51.697 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-04-26T19:51:50.272349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220908,"kb_used_data":4708,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753812,"statfs":{"total":773094113280,"available":772867903488,"internally_reserved":0,"allocated":4820992,"data_stored":2742736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.240652"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050068+0000","last_change":"2024-04-26T19:51:44.057653+0000","last_active":"2024-04-26T19:51:45.050068+0000","last_peered":"2024-04-26T19:51:45.050068+0000","last_clean":"2024-04-26T19:51:45.050068+0000","last_became_active":"2024-04-26T19:51:44.057379+0000","last_became_peered":"2024-04-26T19:51:44.057379+0000","last_unstale":"2024-04-26T19:51:45.050068+0000","last_undegraded":"2024-04-26T19:51:45.050068+0000","last_fullsized":"2024-04-26T19:51:45.050068+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:51.773418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049871+0000","last_change":"2024-04-26T19:51:44.052950+0000","last_active":"2024-04-26T19:51:45.049871+0000","last_peered":"2024-04-26T19:51:45.049871+0000","last_clean":"2024-04-26T19:51:45.049871+0000","last_became_active":"2024-04-26T19:51:44.052670+0000","last_became_peered":"2024-04-26T19:51:44.052670+0000","last_unstale":"2024-04-26T19:51:45.049871+0000","last_undegraded":"2024-04-26T19:51:45.049871+0000","last_fullsized":"2024-04-26T19:51:45.049871+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:48.774437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060529+0000","last_change":"2024-04-26T19:51:44.056315+0000","last_active":"2024-04-26T19:51:44.060529+0000","last_peered":"2024-04-26T19:51:44.060529+0000","last_clean":"2024-04-26T19:51:44.060529+0000","last_became_active":"2024-04-26T19:51:44.056068+0000","last_became_peered":"2024-04-26T19:51:44.056068+0000","last_unstale":"2024-04-26T19:51:44.060529+0000","last_undegraded":"2024-04-26T19:51:44.060529+0000","last_fullsized":"2024-04-26T19:51:44.060529+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:37.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645804+0000","last_change":"2024-04-26T19:51:44.051583+0000","last_active":"2024-04-26T19:51:45.645804+0000","last_peered":"2024-04-26T19:51:45.645804+0000","last_clean":"2024-04-26T19:51:45.645804+0000","last_became_active":"2024-04-26T19:51:44.051303+0000","last_became_peered":"2024-04-26T19:51:44.051303+0000","last_unstale":"2024-04-26T19:51:45.645804+0000","last_undegraded":"2024-04-26T19:51:45.645804+0000","last_fullsized":"2024-04-26T19:51:45.645804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:26.452557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645631+0000","last_change":"2024-04-26T19:51:44.047449+0000","last_active":"2024-04-26T19:51:45.645631+0000","last_peered":"2024-04-26T19:51:45.645631+0000","last_clean":"2024-04-26T19:51:45.645631+0000","last_became_active":"2024-04-26T19:51:44.047217+0000","last_became_peered":"2024-04-26T19:51:44.047217+0000","last_unstale":"2024-04-26T19:51:45.645631+0000","last_undegraded":"2024-04-26T19:51:45.645631+0000","last_fullsized":"2024-04-26T19:51:45.645631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:44.843944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.065591+0000","last_change":"2024-04-26T19:51:44.061255+0000","last_active":"2024-04-26T19:51:44.065591+0000","last_peered":"2024-04-26T19:51:44.065591+0000","last_clean":"2024-04-26T19:51:44.065591+0000","last_became_active":"2024-04-26T19:51:44.060988+0000","last_became_peered":"2024-04-26T19:51:44.060988+0000","last_unstale":"2024-04-26T19:51:44.065591+0000","last_undegraded":"2024-04-26T19:51:44.065591+0000","last_fullsized":"2024-04-26T19:51:44.065591+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:05:02.150722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038496+0000","last_change":"2024-04-26T19:51:44.054384+0000","last_active":"2024-04-26T19:51:46.038496+0000","last_peered":"2024-04-26T19:51:46.038496+0000","last_clean":"2024-04-26T19:51:46.038496+0000","last_became_active":"2024-04-26T19:51:44.054112+0000","last_became_peered":"2024-04-26T19:51:44.054112+0000","last_unstale":"2024-04-26T19:51:46.038496+0000","last_undegraded":"2024-04-26T19:51:46.038496+0000","last_fullsized":"2024-04-26T19:51:46.038496+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:28.264211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049759+0000","last_change":"2024-04-26T19:51:44.052078+0000","last_active":"2024-04-26T19:51:45.049759+0000","last_peered":"2024-04-26T19:51:45.049759+0000","last_clean":"2024-04-26T19:51:45.049759+0000","last_became_active":"2024-04-26T19:51:44.051840+0000","last_became_peered":"2024-04-26T19:51:44.051840+0000","last_unstale":"2024-04-26T19:51:45.049759+0000","last_undegraded":"2024-04-26T19:51:45.049759+0000","last_fullsized":"2024-04-26T19:51:45.049759+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:12:24.342993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060559+0000","last_change":"2024-04-26T19:51:44.055643+0000","last_active":"2024-04-26T19:51:44.060559+0000","last_peered":"2024-04-26T19:51:44.060559+0000","last_clean":"2024-04-26T19:51:44.060559+0000","last_became_active":"2024-04-26T19:51:44.055380+0000","last_became_peered":"2024-04-26T19:51:44.055380+0000","last_unstale":"2024-04-26T19:51:44.060559+0000","last_undegraded":"2024-04-26T19:51:44.060559+0000","last_fullsized":"2024-04-26T19:51:44.060559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:35.459759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038815+0000","last_change":"2024-04-26T19:51:44.055916+0000","last_active":"2024-04-26T19:51:46.038815+0000","last_peered":"2024-04-26T19:51:46.038815+0000","last_clean":"2024-04-26T19:51:46.038815+0000","last_became_active":"2024-04-26T19:51:44.055080+0000","last_became_peered":"2024-04-26T19:51:44.055080+0000","last_unstale":"2024-04-26T19:51:46.038815+0000","last_undegraded":"2024-04-26T19:51:46.038815+0000","last_fullsized":"2024-04-26T19:51:46.038815+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:26:28.839416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038866+0000","last_change":"2024-04-26T19:51:44.058213+0000","last_active":"2024-04-26T19:51:46.038866+0000","last_peered":"2024-04-26T19:51:46.038866+0000","last_clean":"2024-04-26T19:51:46.038866+0000","last_became_active":"2024-04-26T19:51:44.057958+0000","last_became_peered":"2024-04-26T19:51:44.057958+0000","last_unstale":"2024-04-26T19:51:46.038866+0000","last_undegraded":"2024-04-26T19:51:46.038866+0000","last_fullsized":"2024-04-26T19:51:46.038866+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:49.952445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057282+0000","last_change":"2024-04-26T19:51:44.051551+0000","last_active":"2024-04-26T19:51:44.057282+0000","last_peered":"2024-04-26T19:51:44.057282+0000","last_clean":"2024-04-26T19:51:44.057282+0000","last_became_active":"2024-04-26T19:51:44.051280+0000","last_became_peered":"2024-04-26T19:51:44.051280+0000","last_unstale":"2024-04-26T19:51:44.057282+0000","last_undegraded":"2024-04-26T19:51:44.057282+0000","last_fullsized":"2024-04-26T19:51:44.057282+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:04.576196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.039143+0000","last_change":"2024-04-26T19:51:44.056142+0000","last_active":"2024-04-26T19:51:46.039143+0000","last_peered":"2024-04-26T19:51:46.039143+0000","last_clean":"2024-04-26T19:51:46.039143+0000","last_became_active":"2024-04-26T19:51:44.055535+0000","last_became_peered":"2024-04-26T19:51:44.055535+0000","last_unstale":"2024-04-26T19:51:46.039143+0000","last_undegraded":"2024-04-26T19:51:46.039143+0000","last_fullsized":"2024-04-26T19:51:46.039143+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:22:51.773418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.048874+0000","last_change":"2024-04-26T19:51:44.048345+0000","last_active":"2024-04-26T19:51:45.048874+0000","last_peered":"2024-04-26T19:51:45.048874+0000","last_clean":"2024-04-26T19:51:45.048874+0000","last_became_active":"2024-04-26T19:51:44.048065+0000","last_became_peered":"2024-04-26T19:51:44.048065+0000","last_unstale":"2024-04-26T19:51:45.048874+0000","last_undegraded":"2024-04-26T19:51:45.048874+0000","last_fullsized":"2024-04-26T19:51:45.048874+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:04.576196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038562+0000","last_change":"2024-04-26T19:51:44.056391+0000","last_active":"2024-04-26T19:51:46.038562+0000","last_peered":"2024-04-26T19:51:46.038562+0000","last_clean":"2024-04-26T19:51:46.038562+0000","last_became_active":"2024-04-26T19:51:44.054742+0000","last_became_peered":"2024-04-26T19:51:44.054742+0000","last_unstale":"2024-04-26T19:51:46.038562+0000","last_undegraded":"2024-04-26T19:51:46.038562+0000","last_fullsized":"2024-04-26T19:51:46.038562+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:42.866288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182149+0000","last_change":"2024-04-26T19:51:44.055994+0000","last_active":"2024-04-26T19:51:46.182149+0000","last_peered":"2024-04-26T19:51:46.182149+0000","last_clean":"2024-04-26T19:51:46.182149+0000","last_became_active":"2024-04-26T19:51:44.055734+0000","last_became_peered":"2024-04-26T19:51:44.055734+0000","last_unstale":"2024-04-26T19:51:46.182149+0000","last_undegraded":"2024-04-26T19:51:46.182149+0000","last_fullsized":"2024-04-26T19:51:46.182149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:37.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182713+0000","last_change":"2024-04-26T19:51:44.055909+0000","last_active":"2024-04-26T19:51:46.182713+0000","last_peered":"2024-04-26T19:51:46.182713+0000","last_clean":"2024-04-26T19:51:46.182713+0000","last_became_active":"2024-04-26T19:51:44.055565+0000","last_became_peered":"2024-04-26T19:51:44.055565+0000","last_unstale":"2024-04-26T19:51:46.182713+0000","last_undegraded":"2024-04-26T19:51:46.182713+0000","last_fullsized":"2024-04-26T19:51:46.182713+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:59:42.866288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057336+0000","last_change":"2024-04-26T19:51:44.048623+0000","last_active":"2024-04-26T19:51:44.057336+0000","last_peered":"2024-04-26T19:51:44.057336+0000","last_clean":"2024-04-26T19:51:44.057336+0000","last_became_active":"2024-04-26T19:51:44.048361+0000","last_became_peered":"2024-04-26T19:51:44.048361+0000","last_unstale":"2024-04-26T19:51:44.057336+0000","last_undegraded":"2024-04-26T19:51:44.057336+0000","last_fullsized":"2024-04-26T19:51:44.057336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:44.843944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645798+0000","last_change":"2024-04-26T19:51:44.048683+0000","last_active":"2024-04-26T19:51:45.645798+0000","last_peered":"2024-04-26T19:51:45.645798+0000","last_clean":"2024-04-26T19:51:45.645798+0000","last_became_active":"2024-04-26T19:51:44.048392+0000","last_became_peered":"2024-04-26T19:51:44.048392+0000","last_unstale":"2024-04-26T19:51:45.645798+0000","last_undegraded":"2024-04-26T19:51:45.645798+0000","last_fullsized":"2024-04-26T19:51:45.645798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:42.436894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.058371+0000","last_change":"2024-04-26T19:51:44.049275+0000","last_active":"2024-04-26T19:51:44.058371+0000","last_peered":"2024-04-26T19:51:44.058371+0000","last_clean":"2024-04-26T19:51:44.058371+0000","last_became_active":"2024-04-26T19:51:44.048962+0000","last_became_peered":"2024-04-26T19:51:44.048962+0000","last_unstale":"2024-04-26T19:51:44.058371+0000","last_undegraded":"2024-04-26T19:51:44.058371+0000","last_fullsized":"2024-04-26T19:51:44.058371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:42.436894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038601+0000","last_change":"2024-04-26T19:51:44.054992+0000","last_active":"2024-04-26T19:51:46.038601+0000","last_peered":"2024-04-26T19:51:46.038601+0000","last_clean":"2024-04-26T19:51:46.038601+0000","last_became_active":"2024-04-26T19:51:44.054217+0000","last_became_peered":"2024-04-26T19:51:44.054217+0000","last_unstale":"2024-04-26T19:51:46.038601+0000","last_undegraded":"2024-04-26T19:51:46.038601+0000","last_fullsized":"2024-04-26T19:51:46.038601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:51.764577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049324+0000","last_change":"2024-04-26T19:51:44.044045+0000","last_active":"2024-04-26T19:51:45.049324+0000","last_peered":"2024-04-26T19:51:45.049324+0000","last_clean":"2024-04-26T19:51:45.049324+0000","last_became_active":"2024-04-26T19:51:44.043745+0000","last_became_peered":"2024-04-26T19:51:44.043745+0000","last_unstale":"2024-04-26T19:51:45.049324+0000","last_undegraded":"2024-04-26T19:51:45.049324+0000","last_fullsized":"2024-04-26T19:51:45.049324+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:17.443111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182157+0000","last_change":"2024-04-26T19:51:44.051746+0000","last_active":"2024-04-26T19:51:46.182157+0000","last_peered":"2024-04-26T19:51:46.182157+0000","last_clean":"2024-04-26T19:51:46.182157+0000","last_became_active":"2024-04-26T19:51:44.051444+0000","last_became_peered":"2024-04-26T19:51:44.051444+0000","last_unstale":"2024-04-26T19:51:46.182157+0000","last_undegraded":"2024-04-26T19:51:46.182157+0000","last_fullsized":"2024-04-26T19:51:46.182157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:46.511485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645598+0000","last_change":"2024-04-26T19:51:44.044956+0000","last_active":"2024-04-26T19:51:45.645598+0000","last_peered":"2024-04-26T19:51:45.645598+0000","last_clean":"2024-04-26T19:51:45.645598+0000","last_became_active":"2024-04-26T19:51:44.044694+0000","last_became_peered":"2024-04-26T19:51:44.044694+0000","last_unstale":"2024-04-26T19:51:45.645598+0000","last_undegraded":"2024-04-26T19:51:45.645598+0000","last_fullsized":"2024-04-26T19:51:45.645598+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:25.346009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057082+0000","last_change":"2024-04-26T19:51:44.042636+0000","last_active":"2024-04-26T19:51:44.057082+0000","last_peered":"2024-04-26T19:51:44.057082+0000","last_clean":"2024-04-26T19:51:44.057082+0000","last_became_active":"2024-04-26T19:51:44.042347+0000","last_became_peered":"2024-04-26T19:51:44.042347+0000","last_unstale":"2024-04-26T19:51:44.057082+0000","last_undegraded":"2024-04-26T19:51:44.057082+0000","last_fullsized":"2024-04-26T19:51:44.057082+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:25.346009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060552+0000","last_change":"2024-04-26T19:51:44.056686+0000","last_active":"2024-04-26T19:51:44.060552+0000","last_peered":"2024-04-26T19:51:44.060552+0000","last_clean":"2024-04-26T19:51:44.060552+0000","last_became_active":"2024-04-26T19:51:44.056412+0000","last_became_peered":"2024-04-26T19:51:44.056412+0000","last_unstale":"2024-04-26T19:51:44.060552+0000","last_undegraded":"2024-04-26T19:51:44.060552+0000","last_fullsized":"2024-04-26T19:51:44.060552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:48.006221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.046013+0000","last_change":"2024-04-26T19:51:44.038799+0000","last_active":"2024-04-26T19:51:44.046013+0000","last_peered":"2024-04-26T19:51:44.046013+0000","last_clean":"2024-04-26T19:51:44.046013+0000","last_became_active":"2024-04-26T19:51:44.038506+0000","last_became_peered":"2024-04-26T19:51:44.038506+0000","last_unstale":"2024-04-26T19:51:44.046013+0000","last_undegraded":"2024-04-26T19:51:44.046013+0000","last_fullsized":"2024-04-26T19:51:44.046013+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:29.958942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038848+0000","last_change":"2024-04-26T19:51:44.060010+0000","last_active":"2024-04-26T19:51:46.038848+0000","last_peered":"2024-04-26T19:51:46.038848+0000","last_clean":"2024-04-26T19:51:46.038848+0000","last_became_active":"2024-04-26T19:51:44.058543+0000","last_became_peered":"2024-04-26T19:51:44.058543+0000","last_unstale":"2024-04-26T19:51:46.038848+0000","last_undegraded":"2024-04-26T19:51:46.038848+0000","last_fullsized":"2024-04-26T19:51:46.038848+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:46:48.774437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.061361+0000","last_change":"2024-04-26T19:51:44.059230+0000","last_active":"2024-04-26T19:51:44.061361+0000","last_peered":"2024-04-26T19:51:44.061361+0000","last_clean":"2024-04-26T19:51:44.061361+0000","last_became_active":"2024-04-26T19:51:44.058955+0000","last_became_peered":"2024-04-26T19:51:44.058955+0000","last_unstale":"2024-04-26T19:51:44.061361+0000","last_undegraded":"2024-04-26T19:51:44.061361+0000","last_fullsized":"2024-04-26T19:51:44.061361+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:21.782979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.061395+0000","last_change":"2024-04-26T19:51:44.051651+0000","last_active":"2024-04-26T19:51:44.061395+0000","last_peered":"2024-04-26T19:51:44.061395+0000","last_clean":"2024-04-26T19:51:44.061395+0000","last_became_active":"2024-04-26T19:51:44.051379+0000","last_became_peered":"2024-04-26T19:51:44.051379+0000","last_unstale":"2024-04-26T19:51:44.061395+0000","last_undegraded":"2024-04-26T19:51:44.061395+0000","last_fullsized":"2024-04-26T19:51:44.061395+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:26.452557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182493+0000","last_change":"2024-04-26T19:51:44.052620+0000","last_active":"2024-04-26T19:51:46.182493+0000","last_peered":"2024-04-26T19:51:46.182493+0000","last_clean":"2024-04-26T19:51:46.182493+0000","last_became_active":"2024-04-26T19:51:44.052131+0000","last_became_peered":"2024-04-26T19:51:44.052131+0000","last_unstale":"2024-04-26T19:51:46.182493+0000","last_undegraded":"2024-04-26T19:51:46.182493+0000","last_fullsized":"2024-04-26T19:51:46.182493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:24.558027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.056621+0000","last_change":"2024-04-26T19:51:44.052591+0000","last_active":"2024-04-26T19:51:44.056621+0000","last_peered":"2024-04-26T19:51:44.056621+0000","last_clean":"2024-04-26T19:51:44.056621+0000","last_became_active":"2024-04-26T19:51:44.052369+0000","last_became_peered":"2024-04-26T19:51:44.052369+0000","last_unstale":"2024-04-26T19:51:44.056621+0000","last_undegraded":"2024-04-26T19:51:44.056621+0000","last_fullsized":"2024-04-26T19:51:44.056621+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:09:10.619763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182474+0000","last_change":"2024-04-26T19:51:44.056559+0000","last_active":"2024-04-26T19:51:46.182474+0000","last_peered":"2024-04-26T19:51:46.182474+0000","last_clean":"2024-04-26T19:51:46.182474+0000","last_became_active":"2024-04-26T19:51:44.056169+0000","last_became_peered":"2024-04-26T19:51:44.056169+0000","last_unstale":"2024-04-26T19:51:46.182474+0000","last_undegraded":"2024-04-26T19:51:46.182474+0000","last_fullsized":"2024-04-26T19:51:46.182474+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:01.488308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.058285+0000","last_change":"2024-04-26T19:51:43.047367+0000","last_active":"2024-04-26T19:51:44.058285+0000","last_peered":"2024-04-26T19:51:44.058285+0000","last_clean":"2024-04-26T19:51:44.058285+0000","last_became_active":"2024-04-26T19:51:43.047075+0000","last_became_peered":"2024-04-26T19:51:43.047075+0000","last_unstale":"2024-04-26T19:51:44.058285+0000","last_undegraded":"2024-04-26T19:51:44.058285+0000","last_fullsized":"2024-04-26T19:51:44.058285+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:37:07.628505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.065585+0000","last_change":"2024-04-26T19:51:44.061149+0000","last_active":"2024-04-26T19:51:44.065585+0000","last_peered":"2024-04-26T19:51:44.065585+0000","last_clean":"2024-04-26T19:51:44.065585+0000","last_became_active":"2024-04-26T19:51:44.060721+0000","last_became_peered":"2024-04-26T19:51:44.060721+0000","last_unstale":"2024-04-26T19:51:44.065585+0000","last_undegraded":"2024-04-26T19:51:44.065585+0000","last_fullsized":"2024-04-26T19:51:44.065585+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:37.856601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.036620+0000","last_change":"2024-04-26T19:51:43.048646+0000","last_active":"2024-04-26T19:51:44.036620+0000","last_peered":"2024-04-26T19:51:44.036620+0000","last_clean":"2024-04-26T19:51:44.036620+0000","last_became_active":"2024-04-26T19:51:43.048370+0000","last_became_peered":"2024-04-26T19:51:43.048370+0000","last_unstale":"2024-04-26T19:51:44.036620+0000","last_undegraded":"2024-04-26T19:51:44.036620+0000","last_fullsized":"2024-04-26T19:51:44.036620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:20.779063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049131+0000","last_change":"2024-04-26T19:51:44.048401+0000","last_active":"2024-04-26T19:51:45.049131+0000","last_peered":"2024-04-26T19:51:45.049131+0000","last_clean":"2024-04-26T19:51:45.049131+0000","last_became_active":"2024-04-26T19:51:44.048106+0000","last_became_peered":"2024-04-26T19:51:44.048106+0000","last_unstale":"2024-04-26T19:51:45.049131+0000","last_undegraded":"2024-04-26T19:51:45.049131+0000","last_fullsized":"2024-04-26T19:51:45.049131+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:46.511485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060843+0000","last_change":"2024-04-26T19:51:43.045945+0000","last_active":"2024-04-26T19:51:44.060843+0000","last_peered":"2024-04-26T19:51:44.060843+0000","last_clean":"2024-04-26T19:51:44.060843+0000","last_became_active":"2024-04-26T19:51:43.045693+0000","last_became_peered":"2024-04-26T19:51:43.045693+0000","last_unstale":"2024-04-26T19:51:44.060843+0000","last_undegraded":"2024-04-26T19:51:44.060843+0000","last_fullsized":"2024-04-26T19:51:44.060843+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:20.779063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050078+0000","last_change":"2024-04-26T19:51:44.050436+0000","last_active":"2024-04-26T19:51:45.050078+0000","last_peered":"2024-04-26T19:51:45.050078+0000","last_clean":"2024-04-26T19:51:45.050078+0000","last_became_active":"2024-04-26T19:51:44.050188+0000","last_became_peered":"2024-04-26T19:51:44.050188+0000","last_unstale":"2024-04-26T19:51:45.050078+0000","last_undegraded":"2024-04-26T19:51:45.050078+0000","last_fullsized":"2024-04-26T19:51:45.050078+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:15.189311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049071+0000","last_change":"2024-04-26T19:51:43.044349+0000","last_active":"2024-04-26T19:51:45.049071+0000","last_peered":"2024-04-26T19:51:45.049071+0000","last_clean":"2024-04-26T19:51:45.049071+0000","last_became_active":"2024-04-26T19:51:43.044048+0000","last_became_peered":"2024-04-26T19:51:43.044048+0000","last_unstale":"2024-04-26T19:51:45.049071+0000","last_undegraded":"2024-04-26T19:51:45.049071+0000","last_fullsized":"2024-04-26T19:51:45.049071+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:41.432978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057167+0000","last_change":"2024-04-26T19:51:44.048846+0000","last_active":"2024-04-26T19:51:44.057167+0000","last_peered":"2024-04-26T19:51:44.057167+0000","last_clean":"2024-04-26T19:51:44.057167+0000","last_became_active":"2024-04-26T19:51:44.048593+0000","last_became_peered":"2024-04-26T19:51:44.048593+0000","last_unstale":"2024-04-26T19:51:44.057167+0000","last_undegraded":"2024-04-26T19:51:44.057167+0000","last_fullsized":"2024-04-26T19:51:44.057167+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:37.856601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038684+0000","last_change":"2024-04-26T19:51:43.050469+0000","last_active":"2024-04-26T19:51:46.038684+0000","last_peered":"2024-04-26T19:51:46.038684+0000","last_clean":"2024-04-26T19:51:46.038684+0000","last_became_active":"2024-04-26T19:51:43.050200+0000","last_became_peered":"2024-04-26T19:51:43.050200+0000","last_unstale":"2024-04-26T19:51:46.038684+0000","last_undegraded":"2024-04-26T19:51:46.038684+0000","last_fullsized":"2024-04-26T19:51:46.038684+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:09:09.615847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038770+0000","last_change":"2024-04-26T19:51:44.054330+0000","last_active":"2024-04-26T19:51:46.038770+0000","last_peered":"2024-04-26T19:51:46.038770+0000","last_clean":"2024-04-26T19:51:46.038770+0000","last_became_active":"2024-04-26T19:51:44.054031+0000","last_became_peered":"2024-04-26T19:51:44.054031+0000","last_unstale":"2024-04-26T19:51:46.038770+0000","last_undegraded":"2024-04-26T19:51:46.038770+0000","last_fullsized":"2024-04-26T19:51:46.038770+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:06:46.770763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182097+0000","last_change":"2024-04-26T19:51:43.047611+0000","last_active":"2024-04-26T19:51:46.182097+0000","last_peered":"2024-04-26T19:51:46.182097+0000","last_clean":"2024-04-26T19:51:46.182097+0000","last_became_active":"2024-04-26T19:51:43.047326+0000","last_became_peered":"2024-04-26T19:51:43.047326+0000","last_unstale":"2024-04-26T19:51:46.182097+0000","last_undegraded":"2024-04-26T19:51:46.182097+0000","last_fullsized":"2024-04-26T19:51:46.182097+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:24.342092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049332+0000","last_change":"2024-04-26T19:51:44.059262+0000","last_active":"2024-04-26T19:51:45.049332+0000","last_peered":"2024-04-26T19:51:45.049332+0000","last_clean":"2024-04-26T19:51:45.049332+0000","last_became_active":"2024-04-26T19:51:44.058996+0000","last_became_peered":"2024-04-26T19:51:44.058996+0000","last_unstale":"2024-04-26T19:51:45.049332+0000","last_undegraded":"2024-04-26T19:51:45.049332+0000","last_fullsized":"2024-04-26T19:51:45.049332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:00:24.558027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.048058+0000","last_change":"2024-04-26T19:51:43.048540+0000","last_active":"2024-04-26T19:51:44.048058+0000","last_peered":"2024-04-26T19:51:44.048058+0000","last_clean":"2024-04-26T19:51:44.048058+0000","last_became_active":"2024-04-26T19:51:43.048278+0000","last_became_peered":"2024-04-26T19:51:43.048278+0000","last_unstale":"2024-04-26T19:51:44.048058+0000","last_undegraded":"2024-04-26T19:51:44.048058+0000","last_fullsized":"2024-04-26T19:51:44.048058+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:41.432978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050068+0000","last_change":"2024-04-26T19:51:44.051602+0000","last_active":"2024-04-26T19:51:45.050068+0000","last_peered":"2024-04-26T19:51:45.050068+0000","last_clean":"2024-04-26T19:51:45.050068+0000","last_became_active":"2024-04-26T19:51:44.051324+0000","last_became_peered":"2024-04-26T19:51:44.051324+0000","last_unstale":"2024-04-26T19:51:45.050068+0000","last_undegraded":"2024-04-26T19:51:45.050068+0000","last_fullsized":"2024-04-26T19:51:45.050068+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:31.097455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.048601+0000","last_change":"2024-04-26T19:51:43.049668+0000","last_active":"2024-04-26T19:51:45.048601+0000","last_peered":"2024-04-26T19:51:45.048601+0000","last_clean":"2024-04-26T19:51:45.048601+0000","last_became_active":"2024-04-26T19:51:43.049389+0000","last_became_peered":"2024-04-26T19:51:43.049389+0000","last_unstale":"2024-04-26T19:51:45.048601+0000","last_undegraded":"2024-04-26T19:51:45.048601+0000","last_fullsized":"2024-04-26T19:51:45.048601+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:43.840028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645839+0000","last_change":"2024-04-26T19:51:44.051971+0000","last_active":"2024-04-26T19:51:45.645839+0000","last_peered":"2024-04-26T19:51:45.645839+0000","last_clean":"2024-04-26T19:51:45.645839+0000","last_became_active":"2024-04-26T19:51:44.051694+0000","last_became_peered":"2024-04-26T19:51:44.051694+0000","last_unstale":"2024-04-26T19:51:45.645839+0000","last_undegraded":"2024-04-26T19:51:45.645839+0000","last_fullsized":"2024-04-26T19:51:45.645839+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:21.782979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050132+0000","last_change":"2024-04-26T19:51:43.046804+0000","last_active":"2024-04-26T19:51:45.050132+0000","last_peered":"2024-04-26T19:51:45.050132+0000","last_clean":"2024-04-26T19:51:45.050132+0000","last_became_active":"2024-04-26T19:51:43.046450+0000","last_became_peered":"2024-04-26T19:51:43.046450+0000","last_unstale":"2024-04-26T19:51:45.050132+0000","last_undegraded":"2024-04-26T19:51:45.050132+0000","last_fullsized":"2024-04-26T19:51:45.050132+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:56:03.572279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182501+0000","last_change":"2024-04-26T19:51:44.061012+0000","last_active":"2024-04-26T19:51:46.182501+0000","last_peered":"2024-04-26T19:51:46.182501+0000","last_clean":"2024-04-26T19:51:46.182501+0000","last_became_active":"2024-04-26T19:51:44.060705+0000","last_became_peered":"2024-04-26T19:51:44.060705+0000","last_unstale":"2024-04-26T19:51:46.182501+0000","last_undegraded":"2024-04-26T19:51:46.182501+0000","last_fullsized":"2024-04-26T19:51:46.182501+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:50:56.774722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038930+0000","last_change":"2024-04-26T19:51:43.045862+0000","last_active":"2024-04-26T19:51:46.038930+0000","last_peered":"2024-04-26T19:51:46.038930+0000","last_clean":"2024-04-26T19:51:46.038930+0000","last_became_active":"2024-04-26T19:51:43.045355+0000","last_became_peered":"2024-04-26T19:51:43.045355+0000","last_unstale":"2024-04-26T19:51:46.038930+0000","last_undegraded":"2024-04-26T19:51:46.038930+0000","last_fullsized":"2024-04-26T19:51:46.038930+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:48.512804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057107+0000","last_change":"2024-04-26T19:51:44.044487+0000","last_active":"2024-04-26T19:51:44.057107+0000","last_peered":"2024-04-26T19:51:44.057107+0000","last_clean":"2024-04-26T19:51:44.057107+0000","last_became_active":"2024-04-26T19:51:44.044242+0000","last_became_peered":"2024-04-26T19:51:44.044242+0000","last_unstale":"2024-04-26T19:51:44.057107+0000","last_undegraded":"2024-04-26T19:51:44.057107+0000","last_fullsized":"2024-04-26T19:51:44.057107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:49.516721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038934+0000","last_change":"2024-04-26T19:51:43.045767+0000","last_active":"2024-04-26T19:51:46.038934+0000","last_peered":"2024-04-26T19:51:46.038934+0000","last_clean":"2024-04-26T19:51:46.038934+0000","last_became_active":"2024-04-26T19:51:43.045410+0000","last_became_peered":"2024-04-26T19:51:43.045410+0000","last_unstale":"2024-04-26T19:51:46.038934+0000","last_undegraded":"2024-04-26T19:51:46.038934+0000","last_fullsized":"2024-04-26T19:51:46.038934+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:47.002305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049371+0000","last_change":"2024-04-26T19:51:44.045406+0000","last_active":"2024-04-26T19:51:45.049371+0000","last_peered":"2024-04-26T19:51:45.049371+0000","last_clean":"2024-04-26T19:51:45.049371+0000","last_became_active":"2024-04-26T19:51:44.045151+0000","last_became_peered":"2024-04-26T19:51:44.045151+0000","last_unstale":"2024-04-26T19:51:45.049371+0000","last_undegraded":"2024-04-26T19:51:45.049371+0000","last_fullsized":"2024-04-26T19:51:45.049371+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:17.748061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050288+0000","last_change":"2024-04-26T19:51:43.049080+0000","last_active":"2024-04-26T19:51:45.050288+0000","last_peered":"2024-04-26T19:51:45.050288+0000","last_clean":"2024-04-26T19:51:45.050288+0000","last_became_active":"2024-04-26T19:51:43.048811+0000","last_became_peered":"2024-04-26T19:51:43.048811+0000","last_unstale":"2024-04-26T19:51:45.050288+0000","last_undegraded":"2024-04-26T19:51:45.050288+0000","last_fullsized":"2024-04-26T19:51:45.050288+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:34.455843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057203+0000","last_change":"2024-04-26T19:51:44.049931+0000","last_active":"2024-04-26T19:51:44.057203+0000","last_peered":"2024-04-26T19:51:44.057203+0000","last_clean":"2024-04-26T19:51:44.057203+0000","last_became_active":"2024-04-26T19:51:44.049685+0000","last_became_peered":"2024-04-26T19:51:44.049685+0000","last_unstale":"2024-04-26T19:51:44.057203+0000","last_undegraded":"2024-04-26T19:51:44.057203+0000","last_fullsized":"2024-04-26T19:51:44.057203+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:05:02.150722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050133+0000","last_change":"2024-04-26T19:51:43.050284+0000","last_active":"2024-04-26T19:51:45.050133+0000","last_peered":"2024-04-26T19:51:45.050133+0000","last_clean":"2024-04-26T19:51:45.050133+0000","last_became_active":"2024-04-26T19:51:43.049914+0000","last_became_peered":"2024-04-26T19:51:43.049914+0000","last_unstale":"2024-04-26T19:51:45.050133+0000","last_undegraded":"2024-04-26T19:51:45.050133+0000","last_fullsized":"2024-04-26T19:51:45.050133+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:36.468083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049770+0000","last_change":"2024-04-26T19:51:44.047575+0000","last_active":"2024-04-26T19:51:45.049770+0000","last_peered":"2024-04-26T19:51:45.049770+0000","last_clean":"2024-04-26T19:51:45.049770+0000","last_became_active":"2024-04-26T19:51:44.047306+0000","last_became_peered":"2024-04-26T19:51:44.047306+0000","last_unstale":"2024-04-26T19:51:45.049770+0000","last_undegraded":"2024-04-26T19:51:45.049770+0000","last_fullsized":"2024-04-26T19:51:45.049770+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:26:28.839416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.049689+0000","last_change":"2024-04-26T19:51:43.048681+0000","last_active":"2024-04-26T19:51:44.049689+0000","last_peered":"2024-04-26T19:51:44.049689+0000","last_clean":"2024-04-26T19:51:44.049689+0000","last_became_active":"2024-04-26T19:51:43.048395+0000","last_became_peered":"2024-04-26T19:51:43.048395+0000","last_unstale":"2024-04-26T19:51:44.049689+0000","last_undegraded":"2024-04-26T19:51:44.049689+0000","last_fullsized":"2024-04-26T19:51:44.049689+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:18.241418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182339+0000","last_change":"2024-04-26T19:51:44.053335+0000","last_active":"2024-04-26T19:51:46.182339+0000","last_peered":"2024-04-26T19:51:46.182339+0000","last_clean":"2024-04-26T19:51:46.182339+0000","last_became_active":"2024-04-26T19:51:44.053064+0000","last_became_peered":"2024-04-26T19:51:44.053064+0000","last_unstale":"2024-04-26T19:51:46.182339+0000","last_undegraded":"2024-04-26T19:51:46.182339+0000","last_fullsized":"2024-04-26T19:51:46.182339+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:22:17.443111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.034754+0000","last_change":"2024-04-26T19:51:43.047181+0000","last_active":"2024-04-26T19:51:44.034754+0000","last_peered":"2024-04-26T19:51:44.034754+0000","last_clean":"2024-04-26T19:51:44.034754+0000","last_became_active":"2024-04-26T19:51:43.046880+0000","last_became_peered":"2024-04-26T19:51:43.046880+0000","last_unstale":"2024-04-26T19:51:44.034754+0000","last_undegraded":"2024-04-26T19:51:44.034754+0000","last_fullsized":"2024-04-26T19:51:44.034754+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:31:44.022728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050059+0000","last_change":"2024-04-26T19:51:44.058112+0000","last_active":"2024-04-26T19:51:45.050059+0000","last_peered":"2024-04-26T19:51:45.050059+0000","last_clean":"2024-04-26T19:51:45.050059+0000","last_became_active":"2024-04-26T19:51:44.057398+0000","last_became_peered":"2024-04-26T19:51:44.057398+0000","last_unstale":"2024-04-26T19:51:45.050059+0000","last_undegraded":"2024-04-26T19:51:45.050059+0000","last_fullsized":"2024-04-26T19:51:45.050059+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:15.775387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182066+0000","last_change":"2024-04-26T19:51:43.049832+0000","last_active":"2024-04-26T19:51:46.182066+0000","last_peered":"2024-04-26T19:51:46.182066+0000","last_clean":"2024-04-26T19:51:46.182066+0000","last_became_active":"2024-04-26T19:51:43.049559+0000","last_became_peered":"2024-04-26T19:51:43.049559+0000","last_unstale":"2024-04-26T19:51:46.182066+0000","last_undegraded":"2024-04-26T19:51:46.182066+0000","last_fullsized":"2024-04-26T19:51:46.182066+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:41:28.955025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050262+0000","last_change":"2024-04-26T19:51:44.051037+0000","last_active":"2024-04-26T19:51:45.050262+0000","last_peered":"2024-04-26T19:51:45.050262+0000","last_clean":"2024-04-26T19:51:45.050262+0000","last_became_active":"2024-04-26T19:51:44.050782+0000","last_became_peered":"2024-04-26T19:51:44.050782+0000","last_unstale":"2024-04-26T19:51:45.050262+0000","last_undegraded":"2024-04-26T19:51:45.050262+0000","last_fullsized":"2024-04-26T19:51:45.050262+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:00:30.533407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.039023+0000","last_change":"2024-04-26T19:51:43.047262+0000","last_active":"2024-04-26T19:51:46.039023+0000","last_peered":"2024-04-26T19:51:46.039023+0000","last_clean":"2024-04-26T19:51:46.039023+0000","last_became_active":"2024-04-26T19:51:43.046905+0000","last_became_peered":"2024-04-26T19:51:43.046905+0000","last_unstale":"2024-04-26T19:51:46.039023+0000","last_undegraded":"2024-04-26T19:51:46.039023+0000","last_fullsized":"2024-04-26T19:51:46.039023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:36.852685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038931+0000","last_change":"2024-04-26T19:51:44.054753+0000","last_active":"2024-04-26T19:51:46.038931+0000","last_peered":"2024-04-26T19:51:46.038931+0000","last_clean":"2024-04-26T19:51:46.038931+0000","last_became_active":"2024-04-26T19:51:44.054353+0000","last_became_peered":"2024-04-26T19:51:44.054353+0000","last_unstale":"2024-04-26T19:51:46.038931+0000","last_undegraded":"2024-04-26T19:51:46.038931+0000","last_fullsized":"2024-04-26T19:51:46.038931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:16:15.775387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049136+0000","last_change":"2024-04-26T19:51:43.040875+0000","last_active":"2024-04-26T19:51:45.049136+0000","last_peered":"2024-04-26T19:51:45.049136+0000","last_clean":"2024-04-26T19:51:45.049136+0000","last_became_active":"2024-04-26T19:51:43.040590+0000","last_became_peered":"2024-04-26T19:51:43.040590+0000","last_unstale":"2024-04-26T19:51:45.049136+0000","last_undegraded":"2024-04-26T19:51:45.049136+0000","last_fullsized":"2024-04-26T19:51:45.049136+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:20.779063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049028+0000","last_change":"2024-04-26T19:51:44.050029+0000","last_active":"2024-04-26T19:51:45.049028+0000","last_peered":"2024-04-26T19:51:45.049028+0000","last_clean":"2024-04-26T19:51:45.049028+0000","last_became_active":"2024-04-26T19:51:44.049607+0000","last_became_peered":"2024-04-26T19:51:44.049607+0000","last_unstale":"2024-04-26T19:51:45.049028+0000","last_undegraded":"2024-04-26T19:51:45.049028+0000","last_fullsized":"2024-04-26T19:51:45.049028+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:16:37.472000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645695+0000","last_change":"2024-04-26T19:51:43.048029+0000","last_active":"2024-04-26T19:51:45.645695+0000","last_peered":"2024-04-26T19:51:45.645695+0000","last_clean":"2024-04-26T19:51:45.645695+0000","last_became_active":"2024-04-26T19:51:43.047727+0000","last_became_peered":"2024-04-26T19:51:43.047727+0000","last_unstale":"2024-04-26T19:51:45.645695+0000","last_undegraded":"2024-04-26T19:51:45.645695+0000","last_fullsized":"2024-04-26T19:51:45.645695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:28.110640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645903+0000","last_change":"2024-04-26T19:51:44.050733+0000","last_active":"2024-04-26T19:51:45.645903+0000","last_peered":"2024-04-26T19:51:45.645903+0000","last_clean":"2024-04-26T19:51:45.645903+0000","last_became_active":"2024-04-26T19:51:44.050379+0000","last_became_peered":"2024-04-26T19:51:44.050379+0000","last_unstale":"2024-04-26T19:51:45.645903+0000","last_undegraded":"2024-04-26T19:51:45.645903+0000","last_fullsized":"2024-04-26T19:51:45.645903+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:19.245335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.048172+0000","last_change":"2024-04-26T19:51:43.045943+0000","last_active":"2024-04-26T19:51:44.048172+0000","last_peered":"2024-04-26T19:51:44.048172+0000","last_clean":"2024-04-26T19:51:44.048172+0000","last_became_active":"2024-04-26T19:51:43.045562+0000","last_became_peered":"2024-04-26T19:51:43.045562+0000","last_unstale":"2024-04-26T19:51:44.048172+0000","last_undegraded":"2024-04-26T19:51:44.048172+0000","last_fullsized":"2024-04-26T19:51:44.048172+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:25.448641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182580+0000","last_change":"2024-04-26T19:51:44.046440+0000","last_active":"2024-04-26T19:51:46.182580+0000","last_peered":"2024-04-26T19:51:46.182580+0000","last_clean":"2024-04-26T19:51:46.182580+0000","last_became_active":"2024-04-26T19:51:44.045900+0000","last_became_peered":"2024-04-26T19:51:44.045900+0000","last_unstale":"2024-04-26T19:51:46.182580+0000","last_undegraded":"2024-04-26T19:51:46.182580+0000","last_fullsized":"2024-04-26T19:51:46.182580+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:32:51.418205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645593+0000","last_change":"2024-04-26T19:51:43.050100+0000","last_active":"2024-04-26T19:51:45.645593+0000","last_peered":"2024-04-26T19:51:45.645593+0000","last_clean":"2024-04-26T19:51:45.645593+0000","last_became_active":"2024-04-26T19:51:43.049842+0000","last_became_peered":"2024-04-26T19:51:43.049842+0000","last_unstale":"2024-04-26T19:51:45.645593+0000","last_undegraded":"2024-04-26T19:51:45.645593+0000","last_fullsized":"2024-04-26T19:51:45.645593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:15.689198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057568+0000","last_change":"2024-04-26T19:51:44.046563+0000","last_active":"2024-04-26T19:51:44.057568+0000","last_peered":"2024-04-26T19:51:44.057568+0000","last_clean":"2024-04-26T19:51:44.057568+0000","last_became_active":"2024-04-26T19:51:44.046322+0000","last_became_peered":"2024-04-26T19:51:44.046322+0000","last_unstale":"2024-04-26T19:51:44.057568+0000","last_undegraded":"2024-04-26T19:51:44.057568+0000","last_fullsized":"2024-04-26T19:51:44.057568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:48.006221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645580+0000","last_change":"2024-04-26T19:51:43.042457+0000","last_active":"2024-04-26T19:51:45.645580+0000","last_peered":"2024-04-26T19:51:45.645580+0000","last_clean":"2024-04-26T19:51:45.645580+0000","last_became_active":"2024-04-26T19:51:43.042081+0000","last_became_peered":"2024-04-26T19:51:43.042081+0000","last_unstale":"2024-04-26T19:51:45.645580+0000","last_undegraded":"2024-04-26T19:51:45.645580+0000","last_fullsized":"2024-04-26T19:51:45.645580+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:42:14.235749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038755+0000","last_change":"2024-04-26T19:51:44.054784+0000","last_active":"2024-04-26T19:51:46.038755+0000","last_peered":"2024-04-26T19:51:46.038755+0000","last_clean":"2024-04-26T19:51:46.038755+0000","last_became_active":"2024-04-26T19:51:44.054399+0000","last_became_peered":"2024-04-26T19:51:44.054399+0000","last_unstale":"2024-04-26T19:51:46.038755+0000","last_undegraded":"2024-04-26T19:51:46.038755+0000","last_fullsized":"2024-04-26T19:51:46.038755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:12:24.342993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182262+0000","last_change":"2024-04-26T19:51:43.049855+0000","last_active":"2024-04-26T19:51:46.182262+0000","last_peered":"2024-04-26T19:51:46.182262+0000","last_clean":"2024-04-26T19:51:46.182262+0000","last_became_active":"2024-04-26T19:51:43.049519+0000","last_became_peered":"2024-04-26T19:51:43.049519+0000","last_unstale":"2024-04-26T19:51:46.182262+0000","last_undegraded":"2024-04-26T19:51:46.182262+0000","last_fullsized":"2024-04-26T19:51:46.182262+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:25:41.432978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.050183+0000","last_change":"2024-04-26T19:51:44.052190+0000","last_active":"2024-04-26T19:51:45.050183+0000","last_peered":"2024-04-26T19:51:45.050183+0000","last_clean":"2024-04-26T19:51:45.050183+0000","last_became_active":"2024-04-26T19:51:44.051942+0000","last_became_peered":"2024-04-26T19:51:44.051942+0000","last_unstale":"2024-04-26T19:51:45.050183+0000","last_undegraded":"2024-04-26T19:51:45.050183+0000","last_fullsized":"2024-04-26T19:51:45.050183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:53:14.527062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645498+0000","last_change":"2024-04-26T19:51:43.044089+0000","last_active":"2024-04-26T19:51:45.645498+0000","last_peered":"2024-04-26T19:51:45.645498+0000","last_clean":"2024-04-26T19:51:45.645498+0000","last_became_active":"2024-04-26T19:51:43.043839+0000","last_became_peered":"2024-04-26T19:51:43.043839+0000","last_unstale":"2024-04-26T19:51:45.645498+0000","last_undegraded":"2024-04-26T19:51:45.645498+0000","last_fullsized":"2024-04-26T19:51:45.645498+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:26.064251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.060362+0000","last_change":"2024-04-26T19:51:44.056608+0000","last_active":"2024-04-26T19:51:44.060362+0000","last_peered":"2024-04-26T19:51:44.060362+0000","last_clean":"2024-04-26T19:51:44.060362+0000","last_became_active":"2024-04-26T19:51:44.056324+0000","last_became_peered":"2024-04-26T19:51:44.056324+0000","last_unstale":"2024-04-26T19:51:44.060362+0000","last_undegraded":"2024-04-26T19:51:44.060362+0000","last_fullsized":"2024-04-26T19:51:44.060362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:43:49.516721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.039083+0000","last_change":"2024-04-26T19:51:43.046179+0000","last_active":"2024-04-26T19:51:46.039083+0000","last_peered":"2024-04-26T19:51:46.039083+0000","last_clean":"2024-04-26T19:51:46.039083+0000","last_became_active":"2024-04-26T19:51:43.045671+0000","last_became_peered":"2024-04-26T19:51:43.045671+0000","last_unstale":"2024-04-26T19:51:46.039083+0000","last_undegraded":"2024-04-26T19:51:46.039083+0000","last_fullsized":"2024-04-26T19:51:46.039083+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:05:01.146806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182600+0000","last_change":"2024-04-26T19:51:44.061767+0000","last_active":"2024-04-26T19:51:46.182600+0000","last_peered":"2024-04-26T19:51:46.182600+0000","last_clean":"2024-04-26T19:51:46.182600+0000","last_became_active":"2024-04-26T19:51:44.061480+0000","last_became_peered":"2024-04-26T19:51:44.061480+0000","last_unstale":"2024-04-26T19:51:46.182600+0000","last_undegraded":"2024-04-26T19:51:46.182600+0000","last_fullsized":"2024-04-26T19:51:46.182600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:28.264211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.645787+0000","last_change":"2024-04-26T19:51:43.040495+0000","last_active":"2024-04-26T19:51:45.645787+0000","last_peered":"2024-04-26T19:51:45.645787+0000","last_clean":"2024-04-26T19:51:45.645787+0000","last_became_active":"2024-04-26T19:51:43.040233+0000","last_became_peered":"2024-04-26T19:51:43.040233+0000","last_unstale":"2024-04-26T19:51:45.645787+0000","last_undegraded":"2024-04-26T19:51:45.645787+0000","last_fullsized":"2024-04-26T19:51:45.645787+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:55:08.794958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049931+0000","last_change":"2024-04-26T19:51:44.049182+0000","last_active":"2024-04-26T19:51:45.049931+0000","last_peered":"2024-04-26T19:51:45.049931+0000","last_clean":"2024-04-26T19:51:45.049931+0000","last_became_active":"2024-04-26T19:51:44.048926+0000","last_became_peered":"2024-04-26T19:51:44.048926+0000","last_unstale":"2024-04-26T19:51:45.049931+0000","last_undegraded":"2024-04-26T19:51:45.049931+0000","last_fullsized":"2024-04-26T19:51:45.049931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:57:49.952445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.036794+0000","last_change":"2024-04-26T19:51:43.050964+0000","last_active":"2024-04-26T19:51:44.036794+0000","last_peered":"2024-04-26T19:51:44.036794+0000","last_clean":"2024-04-26T19:51:44.036794+0000","last_became_active":"2024-04-26T19:51:43.050731+0000","last_became_peered":"2024-04-26T19:51:43.050731+0000","last_unstale":"2024-04-26T19:51:44.036794+0000","last_undegraded":"2024-04-26T19:51:44.036794+0000","last_fullsized":"2024-04-26T19:51:44.036794+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:09:09.615847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.057223+0000","last_change":"2024-04-26T19:51:44.049391+0000","last_active":"2024-04-26T19:51:44.057223+0000","last_peered":"2024-04-26T19:51:44.057223+0000","last_clean":"2024-04-26T19:51:44.057223+0000","last_became_active":"2024-04-26T19:51:44.049156+0000","last_became_peered":"2024-04-26T19:51:44.049156+0000","last_unstale":"2024-04-26T19:51:44.057223+0000","last_undegraded":"2024-04-26T19:51:44.057223+0000","last_fullsized":"2024-04-26T19:51:44.057223+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:29.530604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.036620+0000","last_change":"2024-04-26T19:51:43.050051+0000","last_active":"2024-04-26T19:51:44.036620+0000","last_peered":"2024-04-26T19:51:44.036620+0000","last_clean":"2024-04-26T19:51:44.036620+0000","last_became_active":"2024-04-26T19:51:43.049770+0000","last_became_peered":"2024-04-26T19:51:43.049770+0000","last_unstale":"2024-04-26T19:51:44.036620+0000","last_undegraded":"2024-04-26T19:51:44.036620+0000","last_fullsized":"2024-04-26T19:51:44.036620+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:18:18.241418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.065596+0000","last_change":"2024-04-26T19:51:44.061915+0000","last_active":"2024-04-26T19:51:44.065596+0000","last_peered":"2024-04-26T19:51:44.065596+0000","last_clean":"2024-04-26T19:51:44.065596+0000","last_became_active":"2024-04-26T19:51:44.061626+0000","last_became_peered":"2024-04-26T19:51:44.061626+0000","last_unstale":"2024-04-26T19:51:44.065596+0000","last_undegraded":"2024-04-26T19:51:44.065596+0000","last_fullsized":"2024-04-26T19:51:44.065596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:29.530604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-26T19:51:44.034556+0000","last_change":"2024-04-26T19:51:43.052188+0000","last_active":"2024-04-26T19:51:44.034556+0000","last_peered":"2024-04-26T19:51:44.034556+0000","last_clean":"2024-04-26T19:51:44.034556+0000","last_became_active":"2024-04-26T19:51:43.051996+0000","last_became_peered":"2024-04-26T19:51:43.051996+0000","last_unstale":"2024-04-26T19:51:44.034556+0000","last_undegraded":"2024-04-26T19:51:44.034556+0000","last_fullsized":"2024-04-26T19:51:44.034556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:19.093912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.048974+0000","last_change":"2024-04-26T19:51:44.049649+0000","last_active":"2024-04-26T19:51:45.048974+0000","last_peered":"2024-04-26T19:51:45.048974+0000","last_clean":"2024-04-26T19:51:45.048974+0000","last_became_active":"2024-04-26T19:51:44.049381+0000","last_became_peered":"2024-04-26T19:51:44.049381+0000","last_unstale":"2024-04-26T19:51:45.048974+0000","last_undegraded":"2024-04-26T19:51:45.048974+0000","last_fullsized":"2024-04-26T19:51:45.048974+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:35:35.459759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.048640+0000","last_change":"2024-04-26T19:51:43.045668+0000","last_active":"2024-04-26T19:51:45.048640+0000","last_peered":"2024-04-26T19:51:45.048640+0000","last_clean":"2024-04-26T19:51:45.048640+0000","last_became_active":"2024-04-26T19:51:43.045318+0000","last_became_peered":"2024-04-26T19:51:43.045318+0000","last_unstale":"2024-04-26T19:51:45.048640+0000","last_undegraded":"2024-04-26T19:51:45.048640+0000","last_fullsized":"2024-04-26T19:51:45.048640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:29:25.448641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182464+0000","last_change":"2024-04-26T19:51:44.060016+0000","last_active":"2024-04-26T19:51:46.182464+0000","last_peered":"2024-04-26T19:51:46.182464+0000","last_clean":"2024-04-26T19:51:46.182464+0000","last_became_active":"2024-04-26T19:51:44.059746+0000","last_became_peered":"2024-04-26T19:51:44.059746+0000","last_unstale":"2024-04-26T19:51:46.182464+0000","last_undegraded":"2024-04-26T19:51:46.182464+0000","last_fullsized":"2024-04-26T19:51:46.182464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:58:17.748061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:45.049722+0000","last_change":"2024-04-26T19:51:43.045649+0000","last_active":"2024-04-26T19:51:45.049722+0000","last_peered":"2024-04-26T19:51:45.049722+0000","last_clean":"2024-04-26T19:51:45.049722+0000","last_became_active":"2024-04-26T19:51:43.045411+0000","last_became_peered":"2024-04-26T19:51:43.045411+0000","last_unstale":"2024-04-26T19:51:45.049722+0000","last_undegraded":"2024-04-26T19:51:45.049722+0000","last_fullsized":"2024-04-26T19:51:45.049722+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:28.526687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.038986+0000","last_change":"2024-04-26T19:51:44.058983+0000","last_active":"2024-04-26T19:51:46.038986+0000","last_peered":"2024-04-26T19:51:46.038986+0000","last_clean":"2024-04-26T19:51:46.038986+0000","last_became_active":"2024-04-26T19:51:44.058667+0000","last_became_peered":"2024-04-26T19:51:44.058667+0000","last_unstale":"2024-04-26T19:51:46.038986+0000","last_undegraded":"2024-04-26T19:51:46.038986+0000","last_fullsized":"2024-04-26T19:51:46.038986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:43.022971+0000","last_clean_scrub_stamp":"2024-04-26T19:51:43.022971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:15:31.097455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182398+0000","last_change":"2024-04-26T19:51:43.048837+0000","last_active":"2024-04-26T19:51:46.182398+0000","last_peered":"2024-04-26T19:51:46.182398+0000","last_clean":"2024-04-26T19:51:46.182398+0000","last_became_active":"2024-04-26T19:51:43.048572+0000","last_became_peered":"2024-04-26T19:51:43.048572+0000","last_unstale":"2024-04-26T19:51:46.182398+0000","last_undegraded":"2024-04-26T19:51:46.182398+0000","last_fullsized":"2024-04-26T19:51:46.182398+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:51:42.019054+0000","last_clean_scrub_stamp":"2024-04-26T19:51:42.019054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:16:43.840028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":205,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T19:51:46.182412+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:51:46.182412+0000","last_peered":"2024-04-26T19:51:46.182412+0000","last_clean":"2024-04-26T19:51:46.182412+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:51:46.182412+0000","last_undegraded":"2024-04-26T19:51:46.182412+0000","last_fullsized":"2024-04-26T19:51:46.182412+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574869,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1060864,"data_stored":789207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":191047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574869,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":458752,"data_stored":197269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":458752,"data_stored":197761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":198803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":197227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":190555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1036288,"data_stored":780867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:51:51.703 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-26T19:51:51.703 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-26T19:51:51.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.841+0000 7f5c0452b640 1 Processor -- start 2024-04-26T19:51:51.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.841+0000 7f5c0452b640 1 -- start start 2024-04-26T19:51:51.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c0452b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5bfc06df80 0x7f5bfc073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c0452b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5bfc073f60 0x7f5bfc15ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c0452b640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bfc06e440 con 0x7f5bfc15ac80 2024-04-26T19:51:51.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c0452b640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f5bfc06fe40 con 0x7f5bfc06df80 2024-04-26T19:51:51.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c0452b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5bfc0743b0 con 0x7f5bfc073f60 2024-04-26T19:51:51.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c01ac6640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5bfc06df80 0x7f5bfc073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.843 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c01ac6640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5bfc06df80 0x7f5bfc073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49068/0 (socket says 172.21.15.79:49068) 2024-04-26T19:51:51.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c01ac6640 1 -- 172.21.15.79:0/3058556758 learned_addr learned my addr 172.21.15.79:0/3058556758 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:51.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c02ac8640 1 --2- 172.21.15.79:0/3058556758 >> v2:172.21.15.83:3300/0 conn(0x7f5bfc073f60 0x7f5bfc15ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c01ac6640 1 -- 172.21.15.79:0/3058556758 >> v2:172.21.15.83:3300/0 conn(0x7f5bfc073f60 msgr2=0x7f5bfc15ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:51.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c01ac6640 1 --2- 172.21.15.79:0/3058556758 >> v2:172.21.15.83:3300/0 conn(0x7f5bfc073f60 0x7f5bfc15ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c01ac6640 1 -- 172.21.15.79:0/3058556758 >> v1:172.21.15.83:6789/0 conn(0x7f5bfc15ac80 legacy=0x7f5bfc15e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:51.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.842+0000 7f5c01ac6640 1 -- 172.21.15.79:0/3058556758 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bfc06e720 con 0x7f5bfc06df80 2024-04-26T19:51:51.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.843+0000 7f5c01ac6640 1 --2- 172.21.15.79:0/3058556758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc06df80 0x7f5bfc073a20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc06f120 tx=0x7f5bec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5fbde27bd93159 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:51.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.843+0000 7f5c012c5640 1 -- 172.21.15.79:0/3058556758 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec037070 con 0x7f5bfc06df80 2024-04-26T19:51:51.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.843+0000 7f5c012c5640 1 -- 172.21.15.79:0/3058556758 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec032440 con 0x7f5bfc06df80 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.843+0000 7f5c012c5640 1 -- 172.21.15.79:0/3058556758 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec032810 con 0x7f5bfc06df80 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.844+0000 7f5c0452b640 1 -- 172.21.15.79:0/3058556758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc06df80 msgr2=0x7f5bfc073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.844+0000 7f5c0452b640 1 --2- 172.21.15.79:0/3058556758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc06df80 0x7f5bfc073a20 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc06f120 tx=0x7f5bec02a1a0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.844+0000 7f5c0452b640 1 -- 172.21.15.79:0/3058556758 shutdown_connections 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.844+0000 7f5c0452b640 1 --2- 172.21.15.79:0/3058556758 >> v2:172.21.15.83:3300/0 conn(0x7f5bfc073f60 0x7f5bfc15ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.844+0000 7f5c0452b640 1 --2- 172.21.15.79:0/3058556758 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc06df80 0x7f5bfc073a20 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.844+0000 7f5c0452b640 1 -- 172.21.15.79:0/3058556758 >> 172.21.15.79:0/3058556758 conn(0x7f5bfc0b2540 msgr2=0x7f5bfc0b2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.844+0000 7f5c0452b640 1 -- 172.21.15.79:0/3058556758 shutdown_connections 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.844+0000 7f5c0452b640 1 -- 172.21.15.79:0/3058556758 wait complete. 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.845+0000 7f5c0452b640 1 Processor -- start 2024-04-26T19:51:51.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.845+0000 7f5c0452b640 1 -- start start 2024-04-26T19:51:51.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c0452b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5bfc06df80 0x7f5bfc145a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c0452b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc073f60 0x7f5bfc071ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c0452b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bfc15ac80 0x7f5bfc0aee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c0452b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc15b060 con 0x7f5bfc073f60 2024-04-26T19:51:51.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c0452b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc06ece0 con 0x7f5bfc15ac80 2024-04-26T19:51:51.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c0452b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f5bfc15b1e0 con 0x7f5bfc06df80 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c01ac6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc073f60 0x7f5bfc071ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c01ac6640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc073f60 0x7f5bfc071ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49082/0 (socket says 172.21.15.79:49082) 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c01ac6640 1 -- 172.21.15.79:0/625237764 learned_addr learned my addr 172.21.15.79:0/625237764 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c02ac8640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bfc15ac80 0x7f5bfc0aee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c022c7640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5bfc06df80 0x7f5bfc145a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c01ac6640 1 -- 172.21.15.79:0/625237764 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5bfc06df80 msgr2=0x7f5bfc145a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c01ac6640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5bfc06df80 0x7f5bfc145a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c01ac6640 1 -- 172.21.15.79:0/625237764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bfc15ac80 msgr2=0x7f5bfc0aee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c01ac6640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bfc15ac80 0x7f5bfc0aee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c01ac6640 1 -- 172.21.15.79:0/625237764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bfc072830 con 0x7f5bfc073f60 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c02ac8640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bfc15ac80 0x7f5bfc0aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.846+0000 7f5c01ac6640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc073f60 0x7f5bfc071ec0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5bec000c00 tx=0x7f5bec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.847+0000 7f5bf2ffd640 1 -- 172.21.15.79:0/625237764 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec041020 con 0x7f5bfc073f60 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.847+0000 7f5bf2ffd640 1 -- 172.21.15.79:0/625237764 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec0306c0 con 0x7f5bfc073f60 2024-04-26T19:51:51.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.847+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bfc072b10 con 0x7f5bfc073f60 2024-04-26T19:51:51.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.847+0000 7f5bf2ffd640 1 -- 172.21.15.79:0/625237764 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec037040 con 0x7f5bfc073f60 2024-04-26T19:51:51.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.847+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bfc0730d0 con 0x7f5bfc073f60 2024-04-26T19:51:51.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.848+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bfc0506c0 con 0x7f5bfc073f60 2024-04-26T19:51:51.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.848+0000 7f5bf2ffd640 1 -- 172.21.15.79:0/625237764 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bec044020 con 0x7f5bfc073f60 2024-04-26T19:51:51.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.848+0000 7f5bf2ffd640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f5be006c8f0 0x7f5be006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:51.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.849+0000 7f5bf2ffd640 1 -- 172.21.15.79:0/625237764 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5bec0ac540 con 0x7f5bfc073f60 2024-04-26T19:51:51.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.849+0000 7f5c022c7640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f5be006c8f0 0x7f5be006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:51.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.849+0000 7f5c022c7640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f5be006c8f0 0x7f5be006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009f20 tx=0x7f5bf80094d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:51.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:51.853+0000 7f5bf2ffd640 1 -- 172.21.15.79:0/625237764 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5bec04b030 con 0x7f5bfc073f60 2024-04-26T19:51:52.101 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.100+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5bfc160470 con 0x7f5bfc073f60 2024-04-26T19:51:52.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.102+0000 7f5bf2ffd640 1 -- 172.21.15.79:0/625237764 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f5bec07a7e0 con 0x7f5bfc073f60 2024-04-26T19:51:52.103 INFO:teuthology.orchestra.run.smithi079.stdout:[client.0] 2024-04-26T19:51:52.103 INFO:teuthology.orchestra.run.smithi079.stdout: key = AQBNBSxmVxwrCBAA6mt2hQ7ZI0Y9b09Qqm6RfQ== 2024-04-26T19:51:52.103 INFO:teuthology.orchestra.run.smithi079.stdout: caps mds = "allow" 2024-04-26T19:51:52.103 INFO:teuthology.orchestra.run.smithi079.stdout: caps mon = "allow r" 2024-04-26T19:51:52.103 INFO:teuthology.orchestra.run.smithi079.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-26T19:51:52.103 INFO:teuthology.orchestra.run.smithi079.stderr:updated caps for client.0 2024-04-26T19:51:52.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f5be006c8f0 msgr2=0x7f5be006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f5be006c8f0 0x7f5be006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5bf8009f20 tx=0x7f5bf80094d0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc073f60 msgr2=0x7f5bfc071ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc073f60 0x7f5bfc071ec0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5bec000c00 tx=0x7f5bec0046c0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 shutdown_connections 2024-04-26T19:51:52.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f5be006c8f0 0x7f5be006edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5bfc15ac80 0x7f5bfc0aee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5bfc073f60 0x7f5bfc071ec0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 --2- 172.21.15.79:0/625237764 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5bfc06df80 0x7f5bfc145a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 >> 172.21.15.79:0/625237764 conn(0x7f5bfc0b2540 msgr2=0x7f5bfc0b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:52.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 shutdown_connections 2024-04-26T19:51:52.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.104+0000 7f5c0452b640 1 -- 172.21.15.79:0/625237764 wait complete. 2024-04-26T19:51:52.116 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:51:52.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.211+0000 7f54483bf640 1 Processor -- start 2024-04-26T19:51:52.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.211+0000 7f54483bf640 1 -- start start 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f54483bf640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f54400cd9c0 0x7f54400b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f54483bf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f54400735a0 0x7f54400b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f54483bf640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54400ce1a0 con 0x7f5440156670 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f54483bf640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f5440073110 con 0x7f54400cd9c0 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f54483bf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f54400cdec0 con 0x7f54400735a0 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f544595a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f54400cd9c0 0x7f54400b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f544615b640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5440156670 0x7f544015d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53642/0 (socket says 172.21.15.79:53642) 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f544615b640 1 -- 172.21.15.79:0/869343257 learned_addr learned my addr 172.21.15.79:0/869343257 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f544595a640 1 -- 172.21.15.79:0/869343257 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f54400735a0 msgr2=0x7f54400b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f544595a640 1 --2- 172.21.15.79:0/869343257 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f54400735a0 0x7f54400b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f544595a640 1 -- 172.21.15.79:0/869343257 >> v1:172.21.15.79:6789/0 conn(0x7f5440156670 legacy=0x7f544015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f5445159640 1 -- 172.21.15.79:0/869343257 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3708519533 0 0) 0x7f54400ce1a0 con 0x7f5440156670 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.212+0000 7f544595a640 1 -- 172.21.15.79:0/869343257 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5440072ac0 con 0x7f54400cd9c0 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.213+0000 7f544595a640 1 --2- 172.21.15.79:0/869343257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54400cd9c0 0x7f54400b3de0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f54300097a0 tx=0x7f543002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b189d0dbd7b9b6e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.213+0000 7f5445159640 1 -- 172.21.15.79:0/869343257 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430037070 con 0x7f54400cd9c0 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.213+0000 7f5445159640 1 -- 172.21.15.79:0/869343257 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5430032440 con 0x7f54400cd9c0 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.213+0000 7f5445159640 1 -- 172.21.15.79:0/869343257 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5430032810 con 0x7f54400cd9c0 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.213+0000 7f54483bf640 1 -- 172.21.15.79:0/869343257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54400cd9c0 msgr2=0x7f54400b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.213+0000 7f54483bf640 1 --2- 172.21.15.79:0/869343257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54400cd9c0 0x7f54400b3de0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f54300097a0 tx=0x7f543002a180 comp rx=0 tx=0).stop 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.214+0000 7f54483bf640 1 -- 172.21.15.79:0/869343257 shutdown_connections 2024-04-26T19:51:52.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.214+0000 7f54483bf640 1 --2- 172.21.15.79:0/869343257 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f54400735a0 0x7f54400b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.214+0000 7f54483bf640 1 --2- 172.21.15.79:0/869343257 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f54400cd9c0 0x7f54400b3de0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.214+0000 7f54483bf640 1 -- 172.21.15.79:0/869343257 >> 172.21.15.79:0/869343257 conn(0x7f54400b39d0 msgr2=0x7f544006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:52.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.214+0000 7f54483bf640 1 -- 172.21.15.79:0/869343257 shutdown_connections 2024-04-26T19:51:52.217 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.214+0000 7f54483bf640 1 -- 172.21.15.79:0/869343257 wait complete. 2024-04-26T19:51:52.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f54483bf640 1 Processor -- start 2024-04-26T19:51:52.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f54483bf640 1 -- start start 2024-04-26T19:51:52.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f54483bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54400735a0 0x7f544014eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f54483bf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f54400cd9c0 0x7f54401651e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f54483bf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5440156670 0x7f5440165720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f54483bf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5440156bd0 con 0x7f5440156670 2024-04-26T19:51:52.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f54483bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5440156a50 con 0x7f54400735a0 2024-04-26T19:51:52.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f54483bf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f54400cf650 con 0x7f54400cd9c0 2024-04-26T19:51:52.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f544695c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5440156670 0x7f5440165720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.215+0000 7f544695c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5440156670 0x7f5440165720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49104/0 (socket says 172.21.15.79:49104) 2024-04-26T19:51:52.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f544695c640 1 -- 172.21.15.79:0/2250475368 learned_addr learned my addr 172.21.15.79:0/2250475368 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:52.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f544615b640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54400735a0 0x7f544014eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f544595a640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f54400cd9c0 0x7f54401651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f544695c640 1 -- 172.21.15.79:0/2250475368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f54400cd9c0 msgr2=0x7f54401651e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f544695c640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f54400cd9c0 0x7f54401651e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f544695c640 1 -- 172.21.15.79:0/2250475368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54400735a0 msgr2=0x7f544014eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f544695c640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54400735a0 0x7f544014eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f544695c640 1 -- 172.21.15.79:0/2250475368 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54400703d0 con 0x7f5440156670 2024-04-26T19:51:52.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f544695c640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5440156670 0x7f5440165720 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f543800b970 tx=0x7f543800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f5436ffd640 1 -- 172.21.15.79:0/2250475368 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438019070 con 0x7f5440156670 2024-04-26T19:51:52.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f5436ffd640 1 -- 172.21.15.79:0/2250475368 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f543800cb60 con 0x7f5440156670 2024-04-26T19:51:52.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f5436ffd640 1 -- 172.21.15.79:0/2250475368 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5438014440 con 0x7f5440156670 2024-04-26T19:51:52.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5440166ef0 con 0x7f5440156670 2024-04-26T19:51:52.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.216+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f544006fe10 con 0x7f5440156670 2024-04-26T19:51:52.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.217+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54400506c0 con 0x7f5440156670 2024-04-26T19:51:52.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.218+0000 7f5436ffd640 1 -- 172.21.15.79:0/2250475368 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f543801c070 con 0x7f5440156670 2024-04-26T19:51:52.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.218+0000 7f5436ffd640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f542406c8f0 0x7f542406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.218+0000 7f5436ffd640 1 -- 172.21.15.79:0/2250475368 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f543808b8e0 con 0x7f5440156670 2024-04-26T19:51:52.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.218+0000 7f544615b640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f542406c8f0 0x7f542406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.218+0000 7f544615b640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f542406c8f0 0x7f542406edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f543c007fd0 tx=0x7f543c007de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.222+0000 7f5436ffd640 1 -- 172.21.15.79:0/2250475368 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f54380595c0 con 0x7f5440156670 2024-04-26T19:51:52.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.435+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f544015a040 con 0x7f5440156670 2024-04-26T19:51:52.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.436+0000 7f5436ffd640 1 -- 172.21.15.79:0/2250475368 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f543805d8c0 con 0x7f5440156670 2024-04-26T19:51:52.436 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:51:52.437 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2061494256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":2061494256},{"type":"v1","addr":"172.21.15.83:6835","nonce":2061494256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/4034640420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":4034640420},{"type":"v1","addr":"172.21.15.83:6834","nonce":4034640420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/238749789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":238749789},{"type":"v1","addr":"172.21.15.79:6836","nonce":238749789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:51:43.380019+0000","modified":"2024-04-26T19:51:45.374734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/2334214524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":2334214524},{"type":"v1","addr":"172.21.15.79:6837","nonce":2334214524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5343,"qdb_cluster":[5343]},"id":2}]} 2024-04-26T19:51:52.437 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f542406c8f0 msgr2=0x7f542406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f542406c8f0 0x7f542406edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f543c007fd0 tx=0x7f543c007de0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5440156670 msgr2=0x7f5440165720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5440156670 0x7f5440165720 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f543800b970 tx=0x7f543800be30 comp rx=0 tx=0).stop 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 shutdown_connections 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f542406c8f0 0x7f542406edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5440156670 0x7f5440165720 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f54400cd9c0 0x7f54401651e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 --2- 172.21.15.79:0/2250475368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f54400735a0 0x7f544014eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 >> 172.21.15.79:0/2250475368 conn(0x7f54400b39d0 msgr2=0x7f544015a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 shutdown_connections 2024-04-26T19:51:52.439 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.438+0000 7f54483bf640 1 -- 172.21.15.79:0/2250475368 wait complete. 2024-04-26T19:51:52.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-26T19:51:43.380019+0000', 'modified': '2024-04-26T19:51:45.374734+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5343}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5343': {'gid': 5343, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.79:6837/2334214524', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.79:6835', 'nonce': 2334214524}, {'type': 'v1', 'addr': '172.21.15.79:6837', 'nonce': 2334214524}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5343, 'qdb_cluster': [5343]} 2024-04-26T19:51:52.852 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T19:51:52.852 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-26T19:51:52.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.951+0000 7fa6013af640 1 Processor -- start 2024-04-26T19:51:52.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.951+0000 7fa6013af640 1 -- start start 2024-04-26T19:51:52.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.952+0000 7fa6013af640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc0b44f0 0x7fa5fc166e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.952+0000 7fa6013af640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5fc0ce1a0 con 0x7fa5fc0cd9c0 2024-04-26T19:51:52.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.952+0000 7fa6013af640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5fc0cdec0 con 0x7fa5fc156670 2024-04-26T19:51:52.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.952+0000 7fa6013af640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa5fc073110 con 0x7fa5fc0b44f0 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.952+0000 7fa5fa7fc640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa5fc0cd9c0 0x7fa5fc0b3de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53654/0 (socket says 172.21.15.79:53654) 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.952+0000 7fa5fa7fc640 1 -- 172.21.15.79:0/2672560023 learned_addr learned my addr 172.21.15.79:0/2672560023 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.952+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1547449575 0 0) 0x7fa5fc0ce1a0 con 0x7fa5fc0cd9c0 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.952+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa5e4003520 con 0x7fa5fc0cd9c0 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.952+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc0b44f0 0x7fa5fc166e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 844772623 0 0) 0x7fa5e4003520 con 0x7fa5fc0cd9c0 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc0b44f0 msgr2=0x7fa5fc166e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc0b44f0 0x7fa5fc166e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 >> v1:172.21.15.83:6789/0 conn(0x7fa5fc156670 legacy=0x7fa5fc15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc072ac0 con 0x7fa5fc0cd9c0 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fa5f0001690 con 0x7fa5fc0cd9c0 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 >> v1:172.21.15.79:6789/0 conn(0x7fa5fc0cd9c0 legacy=0x7fa5fc0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5e40039e0 0x7fa5e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc0b44f0 0x7fa5e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e4004d40 0x7fa5e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc073110 con 0x7fa5e4004d40 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5e400baa0 con 0x7fa5e40039e0 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa5e400bca0 con 0x7fa5fc0b44f0 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e4004d40 0x7fa5e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5faffd640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5e40039e0 0x7fa5e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc0b44f0 0x7fa5e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc0b44f0 msgr2=0x7fa5e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc0b44f0 0x7fa5e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5e40039e0 msgr2=0x7fa5e4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5e40039e0 0x7fa5e4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/2672560023 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5e4003520 con 0x7fa5e4004d40 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.953+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc0b44f0 0x7fa5e4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e4004d40 0x7fa5e40051c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc06c6e0 tx=0x7fa5e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e050b8002580e83a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4036070 con 0x7fa5e4004d40 2024-04-26T19:51:52.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f4031400 con 0x7fa5e4004d40 2024-04-26T19:51:52.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa5f9ffb640 1 -- 172.21.15.79:0/2672560023 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4009690 con 0x7fa5e4004d40 2024-04-26T19:51:52.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa6013af640 1 -- 172.21.15.79:0/2672560023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e4004d40 msgr2=0x7fa5e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa6013af640 1 --2- 172.21.15.79:0/2672560023 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5e4004d40 0x7fa5e40051c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc06c6e0 tx=0x7fa5e4005830 comp rx=0 tx=0).stop 2024-04-26T19:51:52.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/2672560023 reap_dead start 2024-04-26T19:51:52.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa6013af640 1 -- 172.21.15.79:0/2672560023 shutdown_connections 2024-04-26T19:51:52.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa6013af640 1 -- 172.21.15.79:0/2672560023 >> 172.21.15.79:0/2672560023 conn(0x7fa5fc0b39d0 msgr2=0x7fa5fc06feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:52.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa6013af640 1 -- 172.21.15.79:0/2672560023 shutdown_connections 2024-04-26T19:51:52.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.954+0000 7fa6013af640 1 -- 172.21.15.79:0/2672560023 wait complete. 2024-04-26T19:51:52.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.956+0000 7fa6013af640 1 Processor -- start 2024-04-26T19:51:52.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.956+0000 7fa6013af640 1 -- start start 2024-04-26T19:51:52.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.956+0000 7fa6013af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5fc13fd90 0x7fa5fc1539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.956+0000 7fa6013af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc155340 0x7fa5fc150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.956+0000 7fa6013af640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc140170 0x7fa5fc151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.956+0000 7fa6013af640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc156bd0 con 0x7fa5fc13fd90 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.956+0000 7fa6013af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc156a50 con 0x7fa5fc155340 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa6013af640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa5fc0735a0 con 0x7fa5fc140170 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5fc13fd90 0x7fa5fc1539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5fc13fd90 0x7fa5fc1539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49130/0 (socket says 172.21.15.79:49130) 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5faffd640 1 -- 172.21.15.79:0/2046169391 learned_addr learned my addr 172.21.15.79:0/2046169391 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc155340 0x7fa5fc150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc140170 0x7fa5fc151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5faffd640 1 -- 172.21.15.79:0/2046169391 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc140170 msgr2=0x7fa5fc151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5faffd640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc140170 0x7fa5fc151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5faffd640 1 -- 172.21.15.79:0/2046169391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc155340 msgr2=0x7fa5fc150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5faffd640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc155340 0x7fa5fc150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5faffd640 1 -- 172.21.15.79:0/2046169391 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc1542a0 con 0x7fa5fc13fd90 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc155340 0x7fa5fc150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5faffd640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5fc13fd90 0x7fa5fc1539e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00f960 tx=0x7fa5ec00fe20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5e37fe640 1 -- 172.21.15.79:0/2046169391 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec003b80 con 0x7fa5fc13fd90 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5e37fe640 1 -- 172.21.15.79:0/2046169391 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5ec018070 con 0x7fa5fc13fd90 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa6013af640 1 -- 172.21.15.79:0/2046169391 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5fc154580 con 0x7fa5fc13fd90 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa5e37fe640 1 -- 172.21.15.79:0/2046169391 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec00d7f0 con 0x7fa5fc13fd90 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.957+0000 7fa6013af640 1 -- 172.21.15.79:0/2046169391 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5fc154b40 con 0x7fa5fc13fd90 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.959+0000 7fa5e37fe640 1 -- 172.21.15.79:0/2046169391 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa5ec00d990 con 0x7fa5fc13fd90 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.959+0000 7fa5e37fe640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa5e806c8f0 0x7fa5e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.959+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa5e806c8f0 0x7fa5e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.959+0000 7fa5e37fe640 1 -- 172.21.15.79:0/2046169391 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa5ec08bde0 con 0x7fa5fc13fd90 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.960+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa5e806c8f0 0x7fa5e806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0001eb0 tx=0x7fa5f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.960+0000 7fa60419d740 1 Processor -- start 2024-04-26T19:51:52.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.960+0000 7fa60419d740 1 -- start start 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa60419d740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd11440 0x55e92fd11820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa60419d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 0x55e92fd2f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa60419d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fce7120 0x55e92fd2d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa60419d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e92fc4b740 con 0x55e92fce7120 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa60419d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e92fbe73a0 con 0x55e92fd2ece0 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa60419d740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x55e92fc47700 con 0x55e92fd11440 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fce7120 0x55e92fd2d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fce7120 0x55e92fd2d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49140/0 (socket says 172.21.15.79:49140) 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/992629490 learned_addr learned my addr 172.21.15.79:0/992629490 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5faffd640 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd11440 0x55e92fd11820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 0x55e92fd2f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/992629490 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd11440 msgr2=0x55e92fd11820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd11440 0x55e92fd11820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.966 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/992629490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 msgr2=0x55e92fd2f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 0x55e92fd2f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/992629490 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92fc03750 con 0x55e92fce7120 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.961+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 0x55e92fd2f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fce7120 0x55e92fd2d3a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4034040 tx=0x7fa5f402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c1e48277fc953b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa5e0ff9640 1 -- 172.21.15.79:0/992629490 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4036070 con 0x55e92fce7120 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa5e0ff9640 1 -- 172.21.15.79:0/992629490 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f4042070 con 0x55e92fce7120 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa5e0ff9640 1 -- 172.21.15.79:0/992629490 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4004240 con 0x55e92fce7120 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa60419d740 1 -- 172.21.15.79:0/992629490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fce7120 msgr2=0x55e92fd2d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa60419d740 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fce7120 0x55e92fd2d3a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa5f4034040 tx=0x7fa5f402e000 comp rx=0 tx=0).stop 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa60419d740 1 -- 172.21.15.79:0/992629490 shutdown_connections 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa60419d740 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fce7120 0x55e92fd2d3a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa60419d740 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 0x55e92fd2f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa60419d740 1 --2- 172.21.15.79:0/992629490 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd11440 0x55e92fd11820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.967 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.962+0000 7fa60419d740 1 -- 172.21.15.79:0/992629490 >> 172.21.15.79:0/992629490 conn(0x55e92fce3a50 msgr2=0x55e92fd2c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:52.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.963+0000 7fa60419d740 1 -- 172.21.15.79:0/992629490 shutdown_connections 2024-04-26T19:51:52.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.963+0000 7fa60419d740 1 -- 172.21.15.79:0/992629490 wait complete. 2024-04-26T19:51:52.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.963+0000 7fa60419d740 1 -- start start 2024-04-26T19:51:52.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.963+0000 7fa60419d740 1 Processor -- start 2024-04-26T19:51:52.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa60419d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd29e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa60419d740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd2ece0 0x55e92fce0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa60419d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd284c0 0x55e92fd26960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa60419d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e92fce7ec0 con 0x55e92fd11440 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa60419d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e92fbd2fe0 con 0x55e92fd284c0 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa60419d740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x55e92fce1980 con 0x55e92fd2ece0 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa60419d740 10 client.0 _ll_register_callbacks cb 0x55e92fbe9150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd29e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd29e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49152/0 (socket says 172.21.15.79:49152) 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5faffd640 1 -- 172.21.15.79:0/119589700 learned_addr learned my addr 172.21.15.79:0/119589700 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd284c0 0x55e92fd26960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd2ece0 0x55e92fce0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5faffd640 1 -- 172.21.15.79:0/119589700 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd2ece0 msgr2=0x55e92fce0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5faffd640 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd2ece0 0x55e92fce0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5faffd640 1 -- 172.21.15.79:0/119589700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd284c0 msgr2=0x55e92fd26960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5faffd640 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd284c0 0x55e92fd26960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5faffd640 1 -- 172.21.15.79:0/119589700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92f95d2d0 con 0x55e92fd11440 2024-04-26T19:51:52.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd284c0 0x55e92fd26960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:51:52.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5faffd640 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd29e20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec059980 tx=0x7fa5ec059410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e42d79353b0f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-26T19:51:52.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/119589700 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec0625d0 con 0x55e92fd11440 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/119589700 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa5ec00dc80 con 0x55e92fd11440 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.964+0000 7fa60419d740 1 -- 172.21.15.79:0/119589700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f4042070 con 0x55e92fd11440 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/119589700 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fa5ec079050 con 0x55e92fd11440 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa60419d740 10 client.5544 fetch_fsmap learned FSMap version 16 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa60419d740 1 -- 172.21.15.79:0/119589700 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55e92fa276b0 con 0x55e92fd11440 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/119589700 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f4042070 con 0x55e92fd11440 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/119589700 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fa5ec059c50 con 0x55e92fd11440 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa60419d740 10 client.5544 fetch_fsmap finished waiting for FSMap version 16 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa60419d740 10 client.5544 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa60419d740 20 client.5544 populate_metadata read hostname 'smithi079' 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa60419d740 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] conn(0x55e92fd23610 0x55e92fd4a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa60419d740 4 client.5544 mds_command: new command op to 5343 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa60419d740 1 -- 172.21.15.79:0/119589700 --> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e92fc59350 con 0x55e92fd23610 2024-04-26T19:51:52.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] conn(0x55e92fd23610 0x55e92fd4a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa5d9ffb640 10 client.5544 ms_handle_connect on v2:172.21.15.79:6835/2334214524 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.965+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] conn(0x55e92fd23610 0x55e92fd4a390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ef098066f59a38b server_cookie=ed785e99bd37aa97 in_seq=0 out_seq=0 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/119589700 <== mds.0 v2:172.21.15.79:6835/2334214524 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e92fc59350 con 0x55e92fd23610 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa5d9ffb640 10 client.5544 handle_command_reply: tid=0 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 client.5544 shutdown 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 -- 172.21.15.79:0/119589700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 msgr2=0x55e92fd29e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd29e20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec059980 tx=0x7fa5ec059410 comp rx=0 tx=0).stop 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 -- 172.21.15.79:0/119589700 shutdown_connections 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] conn(0x55e92fd23610 0x55e92fd4a390 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd284c0 0x55e92fd26960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa5d9ffb640 0 client.5544 ms_handle_reset on v2:172.21.15.79:6835/2334214524 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd2ece0 0x55e92fce0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 --2- 172.21.15.79:0/119589700 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd29e20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.975 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 -- 172.21.15.79:0/119589700 >> 172.21.15.79:0/119589700 conn(0x55e92fce7120 msgr2=0x55e92fd30060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:52.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 -- 172.21.15.79:0/119589700 shutdown_connections 2024-04-26T19:51:52.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 1 -- 172.21.15.79:0/119589700 wait complete. 2024-04-26T19:51:52.976 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.966+0000 7fa60419d740 20 client.5544 trim_cache size 0 max 16384 2024-04-26T19:51:52.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.984+0000 7fa60419d740 1 Processor -- start 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.984+0000 7fa60419d740 1 -- start start 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa60419d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd11440 0x55e92fd276e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa60419d740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd284c0 0x55e92fd14110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa60419d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd2ece0 0x55e92fd14650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa60419d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc201a70 con 0x55e92fd2ece0 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa60419d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc200dd0 con 0x55e92fd11440 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa60419d740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa5fc201420 con 0x55e92fd284c0 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd2ece0 0x55e92fd14650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5fb7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd2ece0 0x55e92fd14650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49156/0 (socket says 172.21.15.79:49156) 2024-04-26T19:51:52.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/1922742340 learned_addr learned my addr 172.21.15.79:0/1922742340 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/1922742340 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd284c0 msgr2=0x55e92fd14110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/1922742340 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd284c0 0x55e92fd14110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/1922742340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd11440 msgr2=0x55e92fd276e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/1922742340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd11440 0x55e92fd276e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5fb7fe640 1 -- 172.21.15.79:0/1922742340 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92fd2eb00 con 0x55e92fd2ece0 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/1922742340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd2ece0 0x55e92fd14650 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc06d5a0 tx=0x7fa5f402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a59a7957ee3b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5dbfff640 1 -- 172.21.15.79:0/1922742340 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4036070 con 0x55e92fd2ece0 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5dbfff640 1 -- 172.21.15.79:0/1922742340 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f404b070 con 0x55e92fd2ece0 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.985+0000 7fa5dbfff640 1 -- 172.21.15.79:0/1922742340 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f40044b0 con 0x55e92fd2ece0 2024-04-26T19:51:52.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 -- 172.21.15.79:0/1922742340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd2ece0 msgr2=0x55e92fd14650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:52.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 --2- 172.21.15.79:0/1922742340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd2ece0 0x55e92fd14650 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc06d5a0 tx=0x7fa5f402f000 comp rx=0 tx=0).stop 2024-04-26T19:51:52.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 -- 172.21.15.79:0/1922742340 shutdown_connections 2024-04-26T19:51:52.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 --2- 172.21.15.79:0/1922742340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd2ece0 0x55e92fd14650 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 --2- 172.21.15.79:0/1922742340 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd284c0 0x55e92fd14110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 --2- 172.21.15.79:0/1922742340 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd11440 0x55e92fd276e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 -- 172.21.15.79:0/1922742340 >> 172.21.15.79:0/1922742340 conn(0x55e92fce7120 msgr2=0x55e92fce0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:52.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 -- 172.21.15.79:0/1922742340 shutdown_connections 2024-04-26T19:51:52.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 -- 172.21.15.79:0/1922742340 wait complete. 2024-04-26T19:51:52.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.986+0000 7fa60419d740 1 -- start start 2024-04-26T19:51:52.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa60419d740 1 Processor -- start 2024-04-26T19:51:52.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa60419d740 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd045d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa60419d740 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd284c0 0x55e92fd14b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa60419d740 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 0x55e92fd150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa60419d740 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x55e92fd23490 con 0x55e92fd11440 2024-04-26T19:51:52.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa60419d740 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x55e92fd2af10 con 0x55e92fd2ece0 2024-04-26T19:51:52.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa60419d740 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x55e92fd23ab0 con 0x55e92fd284c0 2024-04-26T19:51:52.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa60419d740 10 client.0 _ll_register_callbacks cb 0x7fa5ec059800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T19:51:52.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa5faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd045d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa5faffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd045d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49172/0 (socket says 172.21.15.79:49172) 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa5faffd640 1 -- 172.21.15.79:0/1627103058 learned_addr learned my addr 172.21.15.79:0/1627103058 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd284c0 0x55e92fd14b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.987+0000 7fa5fb7fe640 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 0x55e92fd150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5faffd640 1 -- 172.21.15.79:0/1627103058 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd284c0 msgr2=0x55e92fd14b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5faffd640 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd284c0 0x55e92fd14b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5faffd640 1 -- 172.21.15.79:0/1627103058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 msgr2=0x55e92fd150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5faffd640 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 0x55e92fd150a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5faffd640 1 -- 172.21.15.79:0/1627103058 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92fd14360 con 0x55e92fd11440 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5faffd640 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd045d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00f930 tx=0x7fa5ec0012f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7432094031346d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.79:3300/0 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/1627103058 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ec080060 con 0x55e92fd11440 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/1627103058 <== mon.0 v2:172.21.15.79:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa5ec083b50 con 0x55e92fd11440 2024-04-26T19:51:52.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa60419d740 1 -- 172.21.15.79:0/1627103058 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f404b070 con 0x55e92fd11440 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/1627103058 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7fa5ec059530 con 0x55e92fd11440 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa60419d740 10 client.5550 fetch_fsmap learned FSMap version 16 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa60419d740 1 -- 172.21.15.79:0/1627103058 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55e92fd29910 con 0x55e92fd11440 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/1627103058 <== mon.0 v2:172.21.15.79:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f404b070 con 0x55e92fd11440 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/1627103058 <== mon.0 v2:172.21.15.79:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7fa5ec058e40 con 0x55e92fd11440 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa60419d740 10 client.5550 fetch_fsmap finished waiting for FSMap version 16 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa60419d740 10 client.5550 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa60419d740 20 client.5550 populate_metadata read hostname 'smithi079' 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa60419d740 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] conn(0x55e92fdafa60 0x55e92fdb1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:52.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa60419d740 4 client.5550 mds_command: new command op to 5343 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa60419d740 1 -- 172.21.15.79:0/1627103058 --> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] -- command(tid 0: {"prefix": "status"}) -- 0x55e92fc531c0 con 0x55e92fdafa60 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.988+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] conn(0x55e92fdafa60 0x55e92fdb1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa5d9ffb640 10 client.5550 ms_handle_connect on v2:172.21.15.79:6835/2334214524 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa5fa7fc640 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] conn(0x55e92fdafa60 0x55e92fdb1e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f794a5c0a44e55 server_cookie=4052fc4b0f189742 in_seq=0 out_seq=0 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa5d9ffb640 1 -- 172.21.15.79:0/1627103058 <== mds.0 v2:172.21.15.79:6835/2334214524 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e92fc531c0 con 0x55e92fdafa60 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa5d9ffb640 10 client.5550 handle_command_reply: tid=0 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 client.5550 shutdown 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 -- 172.21.15.79:0/1627103058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 msgr2=0x55e92fd045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd045d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00f930 tx=0x7fa5ec0012f0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 -- 172.21.15.79:0/1627103058 shutdown_connections 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.79:6835/2334214524,v1:172.21.15.79:6837/2334214524] conn(0x55e92fdafa60 0x55e92fdb1e50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x55e92fd2ece0 0x55e92fd150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa5d9ffb640 0 client.5550 ms_handle_reset on v2:172.21.15.79:6835/2334214524 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x55e92fd284c0 0x55e92fd14b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.000 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 --2- 172.21.15.79:0/1627103058 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x55e92fd11440 0x55e92fd045d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 -- 172.21.15.79:0/1627103058 >> 172.21.15.79:0/1627103058 conn(0x55e92fce7120 msgr2=0x55e92fd4fa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:53.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 -- 172.21.15.79:0/1627103058 shutdown_connections 2024-04-26T19:51:53.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 1 -- 172.21.15.79:0/1627103058 wait complete. 2024-04-26T19:51:53.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.989+0000 7fa60419d740 20 client.5550 trim_cache size 0 max 16384 2024-04-26T19:51:53.001 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:51:53.001 INFO:teuthology.orchestra.run.smithi079.stdout: "cluster_fsid": "b372f41f-f07b-4764-9bef-bcff62f9a5d2", 2024-04-26T19:51:53.001 INFO:teuthology.orchestra.run.smithi079.stdout: "whoami": 0, 2024-04-26T19:51:53.001 INFO:teuthology.orchestra.run.smithi079.stdout: "id": 5343, 2024-04-26T19:51:53.002 INFO:teuthology.orchestra.run.smithi079.stdout: "want_state": "up:active", 2024-04-26T19:51:53.002 INFO:teuthology.orchestra.run.smithi079.stdout: "state": "up:active", 2024-04-26T19:51:53.002 INFO:teuthology.orchestra.run.smithi079.stdout: "fs_name": "cephfs", 2024-04-26T19:51:53.002 INFO:teuthology.orchestra.run.smithi079.stdout: "rank_uptime": 8.9588967329999996, 2024-04-26T19:51:53.002 INFO:teuthology.orchestra.run.smithi079.stdout: "mdsmap_epoch": 16, 2024-04-26T19:51:53.002 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch": 39, 2024-04-26T19:51:53.002 INFO:teuthology.orchestra.run.smithi079.stdout: "osdmap_epoch_barrier": 39, 2024-04-26T19:51:53.002 INFO:teuthology.orchestra.run.smithi079.stdout: "uptime": 16.047859238000001 2024-04-26T19:51:53.002 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:51:53.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 -- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa5e806c8f0 msgr2=0x7fa5e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:53.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa5e806c8f0 0x7fa5e806edb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0001eb0 tx=0x7fa5f0009040 comp rx=0 tx=0).stop 2024-04-26T19:51:53.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 -- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5fc13fd90 msgr2=0x7fa5fc1539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:53.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5fc13fd90 0x7fa5fc1539e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa5ec00f960 tx=0x7fa5ec00fe20 comp rx=0 tx=0).stop 2024-04-26T19:51:53.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 -- 172.21.15.79:0/2046169391 shutdown_connections 2024-04-26T19:51:53.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa5e806c8f0 0x7fa5e806edb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa5fc140170 0x7fa5fc151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa5fc155340 0x7fa5fc150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 --2- 172.21.15.79:0/2046169391 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa5fc13fd90 0x7fa5fc1539e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 -- 172.21.15.79:0/2046169391 >> 172.21.15.79:0/2046169391 conn(0x7fa5fc0b39d0 msgr2=0x7fa5fc0717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:53.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 -- 172.21.15.79:0/2046169391 shutdown_connections 2024-04-26T19:51:53.005 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:52.990+0000 7fa6013af640 1 -- 172.21.15.79:0/2046169391 wait complete. 2024-04-26T19:51:53.006 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-26T19:51:53.103 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.103+0000 7fb32942c640 1 Processor -- start 2024-04-26T19:51:53.104 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.103+0000 7fb32942c640 1 -- start start 2024-04-26T19:51:53.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb32942c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb3240b64f0 0x7fb324070750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:53.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb32942c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb32415ffa0 0x7fb324160430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:53.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb32942c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb32415f810 con 0x7fb32406e800 2024-04-26T19:51:53.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb32942c640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fb32415fb10 con 0x7fb32415ffa0 2024-04-26T19:51:53.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb32942c640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb32406f7a0 con 0x7fb3240b64f0 2024-04-26T19:51:53.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb3237fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb32415ffa0 0x7fb324160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:53.105 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb3237fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb32415ffa0 0x7fb324160430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49176/0 (socket says 172.21.15.79:49176) 2024-04-26T19:51:53.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb3237fe640 1 -- 172.21.15.79:0/3104578278 learned_addr learned my addr 172.21.15.79:0/3104578278 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:53.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb322ffd640 1 --2- 172.21.15.79:0/3104578278 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb3240b64f0 0x7fb324070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:53.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb3237fe640 1 -- 172.21.15.79:0/3104578278 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb3240b64f0 msgr2=0x7fb324070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:53.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb3237fe640 1 --2- 172.21.15.79:0/3104578278 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb3240b64f0 0x7fb324070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb3237fe640 1 -- 172.21.15.79:0/3104578278 >> v1:172.21.15.83:6789/0 conn(0x7fb32406e800 legacy=0x7fb3240b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:53.106 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.104+0000 7fb3237fe640 1 -- 172.21.15.79:0/3104578278 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32406ec60 con 0x7fb32415ffa0 2024-04-26T19:51:53.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.105+0000 7fb3237fe640 1 --2- 172.21.15.79:0/3104578278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32415ffa0 0x7fb324160430 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009e50 tx=0x7fb31c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9fa700d69c2b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:53.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.105+0000 7fb321ffb640 1 -- 172.21.15.79:0/3104578278 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c036070 con 0x7fb32415ffa0 2024-04-26T19:51:53.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.105+0000 7fb321ffb640 1 -- 172.21.15.79:0/3104578278 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb31c031600 con 0x7fb32415ffa0 2024-04-26T19:51:53.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.105+0000 7fb321ffb640 1 -- 172.21.15.79:0/3104578278 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb31c0319d0 con 0x7fb32415ffa0 2024-04-26T19:51:53.107 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.106+0000 7fb32942c640 1 -- 172.21.15.79:0/3104578278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32415ffa0 msgr2=0x7fb324160430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:53.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.106+0000 7fb32942c640 1 --2- 172.21.15.79:0/3104578278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32415ffa0 0x7fb324160430 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb31c009e50 tx=0x7fb31c02a710 comp rx=0 tx=0).stop 2024-04-26T19:51:53.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.106+0000 7fb32942c640 1 -- 172.21.15.79:0/3104578278 shutdown_connections 2024-04-26T19:51:53.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.106+0000 7fb32942c640 1 --2- 172.21.15.79:0/3104578278 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb32415ffa0 0x7fb324160430 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.106+0000 7fb32942c640 1 --2- 172.21.15.79:0/3104578278 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb3240b64f0 0x7fb324070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.106+0000 7fb32942c640 1 -- 172.21.15.79:0/3104578278 >> 172.21.15.79:0/3104578278 conn(0x7fb32406c770 msgr2=0x7fb32406cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:53.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.106+0000 7fb32942c640 1 -- 172.21.15.79:0/3104578278 shutdown_connections 2024-04-26T19:51:53.108 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.106+0000 7fb32942c640 1 -- 172.21.15.79:0/3104578278 wait complete. 2024-04-26T19:51:53.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.107+0000 7fb32942c640 1 Processor -- start 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.107+0000 7fb32942c640 1 -- start start 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb32942c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb32406e800 0x7fb3241466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb32942c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3240b64f0 0x7fb3240cd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb32942c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb32415ffa0 0x7fb3240cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb32942c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb32406f4c0 con 0x7fb3240b64f0 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb32942c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb32406f1e0 con 0x7fb32406e800 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb32942c640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb3240b3ff0 con 0x7fb32415ffa0 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb3227fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3240b64f0 0x7fb3240cd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb3227fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3240b64f0 0x7fb3240cd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49180/0 (socket says 172.21.15.79:49180) 2024-04-26T19:51:53.111 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb3227fc640 1 -- 172.21.15.79:0/4258565003 learned_addr learned my addr 172.21.15.79:0/4258565003 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb322ffd640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb32406e800 0x7fb3241466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb3237fe640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb32415ffa0 0x7fb3240cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb3227fc640 1 -- 172.21.15.79:0/4258565003 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb32415ffa0 msgr2=0x7fb3240cd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb3227fc640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb32415ffa0 0x7fb3240cd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb3227fc640 1 -- 172.21.15.79:0/4258565003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb32406e800 msgr2=0x7fb3241466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb3227fc640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb32406e800 0x7fb3241466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.108+0000 7fb3227fc640 1 -- 172.21.15.79:0/4258565003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb32415aae0 con 0x7fb3240b64f0 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.109+0000 7fb3227fc640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3240b64f0 0x7fb3240cd3a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c3c0 tx=0x7fb31800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.109+0000 7fb30b7fe640 1 -- 172.21.15.79:0/4258565003 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3180052d0 con 0x7fb3240b64f0 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.109+0000 7fb30b7fe640 1 -- 172.21.15.79:0/4258565003 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb318005490 con 0x7fb3240b64f0 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.109+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb32415adc0 con 0x7fb3240b64f0 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.109+0000 7fb30b7fe640 1 -- 172.21.15.79:0/4258565003 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb318007260 con 0x7fb3240b64f0 2024-04-26T19:51:53.112 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.109+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb32415b380 con 0x7fb3240b64f0 2024-04-26T19:51:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.110+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3240506c0 con 0x7fb3240b64f0 2024-04-26T19:51:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.110+0000 7fb30b7fe640 1 -- 172.21.15.79:0/4258565003 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb318007400 con 0x7fb3240b64f0 2024-04-26T19:51:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.111+0000 7fb30b7fe640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb31006c8f0 0x7fb31006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:53.113 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.111+0000 7fb30b7fe640 1 -- 172.21.15.79:0/4258565003 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fb31808b130 con 0x7fb3240b64f0 2024-04-26T19:51:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.111+0000 7fb322ffd640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb31006c8f0 0x7fb31006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:53.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.111+0000 7fb322ffd640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb31006c8f0 0x7fb31006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb314009aa0 tx=0x7fb314009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:53.116 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.115+0000 7fb30b7fe640 1 -- 172.21.15.79:0/4258565003 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb318058d80 con 0x7fb3240b64f0 2024-04-26T19:51:53.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.326+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb324072c30 con 0x7fb3240b64f0 2024-04-26T19:51:53.327 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.327+0000 7fb30b7fe640 1 -- 172.21.15.79:0/4258565003 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fb31805d080 con 0x7fb3240b64f0 2024-04-26T19:51:53.327 INFO:teuthology.orchestra.run.smithi079.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T19:51:53.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb31006c8f0 msgr2=0x7fb31006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:53.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb31006c8f0 0x7fb31006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb314009aa0 tx=0x7fb314009190 comp rx=0 tx=0).stop 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3240b64f0 msgr2=0x7fb3240cd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3240b64f0 0x7fb3240cd3a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb31800c3c0 tx=0x7fb31800c880 comp rx=0 tx=0).stop 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 shutdown_connections 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fb31006c8f0 0x7fb31006edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb32415ffa0 0x7fb3240cd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb3240b64f0 0x7fb3240cd3a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 --2- 172.21.15.79:0/4258565003 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb32406e800 0x7fb3241466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.328+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 >> 172.21.15.79:0/4258565003 conn(0x7fb32406c770 msgr2=0x7fb324071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.329+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 shutdown_connections 2024-04-26T19:51:53.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53.329+0000 7fb32942c640 1 -- 172.21.15.79:0/4258565003 wait complete. 2024-04-26T19:51:53.340 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-26T19:51:53.340 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T19:51:53.340 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi079.front.sepia.ceph.com 2024-04-26T19:51:53.340 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T19:51:53.340 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T19:51:53.340 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-26T19:51:53.340 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-26T19:51:53.340 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T19:51:53.341 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T19:51:53.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:53.341 DEBUG:teuthology.orchestra.run.smithi079:> ip addr 2024-04-26T19:51:53.356 INFO:teuthology.orchestra.run.smithi079.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T19:51:53.356 INFO:teuthology.orchestra.run.smithi079.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T19:51:53.356 INFO:teuthology.orchestra.run.smithi079.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T19:51:53.356 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:53.356 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 ::1/128 scope host 2024-04-26T19:51:53.356 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:53.356 INFO:teuthology.orchestra.run.smithi079.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:53.356 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6a brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f0 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:d8:a1:6b brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: altname enp7s0f1 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:86 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 0c:c4:7a:8f:ce:87 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: inet 172.21.15.79/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft 42636sec preferred_lft 42636sec 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::5909:6a14:c050:be1a/64 scope link noprefixroute 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 86:d6:7b:d1:94:52 brd ff:ff:ff:ff:ff:ff 2024-04-26T19:51:53.357 INFO:teuthology.orchestra.run.smithi079.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::347f:e9ff:fe5a:5029/64 scope link 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 8a:10:f3:c9:4a:e1 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::8810:f3ff:fec9:4ae1/64 scope link 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout: link/ether 86:d6:7b:d1:94:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout: inet6 fe80::84d6:7bff:fed1:9452/64 scope link 2024-04-26T19:51:53.358 INFO:teuthology.orchestra.run.smithi079.stdout: valid_lft forever preferred_lft forever 2024-04-26T19:51:53.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:53.359 DEBUG:teuthology.orchestra.run.smithi079:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T19:51:53.419 INFO:teuthology.orchestra.run.smithi079.stdout:1 2024-04-26T19:51:53.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:53.421 DEBUG:teuthology.orchestra.run.smithi079:> route 2024-04-26T19:51:53.476 INFO:teuthology.orchestra.run.smithi079.stdout:Kernel IP routing table 2024-04-26T19:51:53.476 INFO:teuthology.orchestra.run.smithi079.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T19:51:53.476 INFO:teuthology.orchestra.run.smithi079.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T19:51:53.476 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T19:51:53.476 INFO:teuthology.orchestra.run.smithi079.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T19:51:53.477 DEBUG:teuthology.orchestra.run.smithi079:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T19:51:53.477 DEBUG:teuthology.orchestra.run.smithi079:> set -e 2024-04-26T19:51:53.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T19:51:53.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T19:51:53.477 DEBUG:teuthology.orchestra.run.smithi079:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T19:51:53.477 DEBUG:teuthology.orchestra.run.smithi079:> ') 2024-04-26T19:51:53.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T19:51:53.593 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:53.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:53.599 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-04-26T19:51:53.614 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T19:51:53.614 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T19:51:53.614 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T19:51:53.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:53.615 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link set brx.0 up 2024-04-26T19:51:53.679 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:53.679 DEBUG:teuthology.orchestra.run.smithi079:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T19:51:53.733 INFO:teuthology.orchestra.run.smithi079.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T19:51:53.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T19:51:53.734 DEBUG:teuthology.orchestra.run.smithi079:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T19:51:53.788 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T19:51:53.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:53.788 DEBUG:teuthology.orchestra.run.smithi079:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-04-26T19:51:53.870 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-04-26T19:51:53.870 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-04-26T19:51:53.870 INFO:teuthology.orchestra.run.smithi079.stdout:invalid new device string format 2024-04-26T19:51:53.878 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-04-26T19:51:53.878 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-04-26T19:51:53.878 INFO:teuthology.orchestra.run.smithi079.stdout:invalid new device string format 2024-04-26T19:51:53.879 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:53.879 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:53.884 INFO:teuthology.orchestra.run.smithi079.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs 2024-04-26T19:51:53.884 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs". 2024-04-26T19:51:53.884 INFO:teuthology.orchestra.run.smithi079.stdout:invalid new device string format 2024-04-26T19:51:53.885 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:3300,172.21.15.83:3300,172.21.15.83:3301" 2024-04-26T19:51:53.885 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T19:51:53.885 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: resolved to: "172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790" 2024-04-26T19:51:53.885 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.79:6789,172.21.15.83:6789,172.21.15.83:6790:/ 2024-04-26T19:51:53.885 INFO:teuthology.orchestra.run.smithi079.stdout:mount.ceph: options "norequire_active_mds,norbytes,snapdirname=.customsnapkernel,name=0,mds_namespace=cephfs,key=0,fsid=b372f41f-f07b-4764-9bef-bcff62f9a5d2" will pass to kernel 2024-04-26T19:51:53.886 INFO:teuthology.orchestra.run.smithi079.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T19:51:53.886 INFO:teuthology.orchestra.run.smithi079.stdout: You just mounted a file system that supports labels which does not 2024-04-26T19:51:53.886 INFO:teuthology.orchestra.run.smithi079.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T19:51:53.886 INFO:teuthology.orchestra.run.smithi079.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T19:51:53.886 INFO:teuthology.orchestra.run.smithi079.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T19:51:53.887 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T19:51:53.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:53.888 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T19:51:53.951 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T19:51:54.041 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T19:51:54.045 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T19:51:54.051 INFO:teuthology.orchestra.run.smithi079.stdout:client5257 2024-04-26T19:51:54.052 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:51:54.052 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:51:54.072 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-04-26T19:51:54.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.210+0000 7f2e3573f640 1 Processor -- start 2024-04-26T19:51:54.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.210+0000 7f2e3573f640 1 -- start start 2024-04-26T19:51:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e3573f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 0x7f2e300d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e3573f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2e3006ff00 0x7f2e300d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e3573f640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2e301488c0 0x7f2e30148d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e3573f640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f2e300d3e60 con 0x7f2e301488c0 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e3573f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2e300b1290 con 0x7f2e3006ff00 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e3573f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f2e300d4090 con 0x7f2e3006fb20 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2f7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2e301488c0 0x7f2e30148d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2f7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2e301488c0 0x7f2e30148d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49194/0 (socket says 172.21.15.79:49194) 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2f7fe640 1 -- 172.21.15.79:0/2650697736 learned_addr learned my addr 172.21.15.79:0/2650697736 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2effd640 1 --2- 172.21.15.79:0/2650697736 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 0x7f2e300d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2e7fc640 1 --2- 172.21.15.79:0/2650697736 >> v2:172.21.15.83:3300/0 conn(0x7f2e3006ff00 0x7f2e300d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2f7fe640 1 -- 172.21.15.79:0/2650697736 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 msgr2=0x7f2e300d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2f7fe640 1 --2- 172.21.15.79:0/2650697736 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 0x7f2e300d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2f7fe640 1 -- 172.21.15.79:0/2650697736 >> v2:172.21.15.83:3300/0 conn(0x7f2e3006ff00 msgr2=0x7f2e300d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2f7fe640 1 --2- 172.21.15.79:0/2650697736 >> v2:172.21.15.83:3300/0 conn(0x7f2e3006ff00 0x7f2e300d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2f7fe640 1 -- 172.21.15.79:0/2650697736 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e300d4370 con 0x7f2e301488c0 2024-04-26T19:51:54.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.211+0000 7f2e2e7fc640 1 --2- 172.21.15.79:0/2650697736 >> v2:172.21.15.83:3300/0 conn(0x7f2e3006ff00 0x7f2e300d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e2f7fe640 1 --2- 172.21.15.79:0/2650697736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e301488c0 0x7f2e30148d30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2e300742b0 tx=0x7f2e2802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c2fab7f8aa6654 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e2dffb640 1 -- 172.21.15.79:0/2650697736 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28036070 con 0x7f2e301488c0 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e2dffb640 1 -- 172.21.15.79:0/2650697736 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e28031600 con 0x7f2e301488c0 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e2dffb640 1 -- 172.21.15.79:0/2650697736 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e280319d0 con 0x7f2e301488c0 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e3573f640 1 -- 172.21.15.79:0/2650697736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e301488c0 msgr2=0x7f2e30148d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e3573f640 1 --2- 172.21.15.79:0/2650697736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e301488c0 0x7f2e30148d30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2e300742b0 tx=0x7f2e2802a730 comp rx=0 tx=0).stop 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e3573f640 1 -- 172.21.15.79:0/2650697736 shutdown_connections 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e3573f640 1 --2- 172.21.15.79:0/2650697736 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e301488c0 0x7f2e30148d30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e3573f640 1 --2- 172.21.15.79:0/2650697736 >> v2:172.21.15.83:3300/0 conn(0x7f2e3006ff00 0x7f2e300d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e3573f640 1 --2- 172.21.15.79:0/2650697736 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 0x7f2e300d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.212+0000 7f2e3573f640 1 -- 172.21.15.79:0/2650697736 >> 172.21.15.79:0/2650697736 conn(0x7f2e30071c30 msgr2=0x7f2e3006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:54.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.213+0000 7f2e3573f640 1 -- 172.21.15.79:0/2650697736 shutdown_connections 2024-04-26T19:51:54.216 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.213+0000 7f2e3573f640 1 -- 172.21.15.79:0/2650697736 wait complete. 2024-04-26T19:51:54.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.214+0000 7f2e3573f640 1 Processor -- start 2024-04-26T19:51:54.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.214+0000 7f2e3573f640 1 -- start start 2024-04-26T19:51:54.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.214+0000 7f2e3573f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 0x7f2e301f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.214+0000 7f2e3573f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e3006ff00 0x7f2e3015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.214+0000 7f2e3573f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e301488c0 0x7f2e301451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e3573f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300d4c10 con 0x7f2e3006ff00 2024-04-26T19:51:54.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e3573f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300d4930 con 0x7f2e301488c0 2024-04-26T19:51:54.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e3573f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f2e30149270 con 0x7f2e3006fb20 2024-04-26T19:51:54.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e3006ff00 0x7f2e3015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.224 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2e7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e3006ff00 0x7f2e3015f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49202/0 (socket says 172.21.15.79:49202) 2024-04-26T19:51:54.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2e7fc640 1 -- 172.21.15.79:0/3641571951 learned_addr learned my addr 172.21.15.79:0/3641571951 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:54.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2effd640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 0x7f2e301f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.226 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2e7fc640 1 -- 172.21.15.79:0/3641571951 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 msgr2=0x7f2e301f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:54.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2e7fc640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 0x7f2e301f5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2e7fc640 1 -- 172.21.15.79:0/3641571951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e301488c0 msgr2=0x7f2e301451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:54.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2e7fc640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e301488c0 0x7f2e301451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2e7fc640 1 -- 172.21.15.79:0/3641571951 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3015f580 con 0x7f2e3006ff00 2024-04-26T19:51:54.227 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e2e7fc640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e3006ff00 0x7f2e3015f040 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400c7f0 tx=0x7f2e2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:54.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e177fe640 1 -- 172.21.15.79:0/3641571951 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24015070 con 0x7f2e3006ff00 2024-04-26T19:51:54.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e177fe640 1 -- 172.21.15.79:0/3641571951 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e24007a30 con 0x7f2e3006ff00 2024-04-26T19:51:54.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e30144ea0 con 0x7f2e3006ff00 2024-04-26T19:51:54.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e177fe640 1 -- 172.21.15.79:0/3641571951 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e24007e00 con 0x7f2e3006ff00 2024-04-26T19:51:54.228 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.215+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e301f5540 con 0x7f2e3006ff00 2024-04-26T19:51:54.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.216+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e300506c0 con 0x7f2e3006ff00 2024-04-26T19:51:54.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.217+0000 7f2e177fe640 1 -- 172.21.15.79:0/3641571951 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2e24018070 con 0x7f2e3006ff00 2024-04-26T19:51:54.229 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.217+0000 7f2e177fe640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e1c06c8f0 0x7f2e1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.217+0000 7f2e177fe640 1 -- 172.21.15.79:0/3641571951 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2e2408bcc0 con 0x7f2e3006ff00 2024-04-26T19:51:54.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.217+0000 7f2e2effd640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e1c06c8f0 0x7f2e1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.230 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.217+0000 7f2e2effd640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e1c06c8f0 0x7f2e1c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2e20006fd0 tx=0x7f2e20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:54.231 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.221+0000 7f2e177fe640 1 -- 172.21.15.79:0/3641571951 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2e24059f60 con 0x7f2e3006ff00 2024-04-26T19:51:54.420 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.420+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f2e300729e0 con 0x7f2e1c06c8f0 2024-04-26T19:51:54.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.421+0000 7f2e177fe640 1 -- 172.21.15.79:0/3641571951 <== mgr.4104 v2:172.21.15.79:6832/3161939478 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f2e300729e0 con 0x7f2e1c06c8f0 2024-04-26T19:51:54.422 INFO:teuthology.orchestra.run.smithi079.stdout:[ 2024-04-26T19:51:54.422 INFO:teuthology.orchestra.run.smithi079.stdout: { 2024-04-26T19:51:54.422 INFO:teuthology.orchestra.run.smithi079.stdout: "name": "cephfs" 2024-04-26T19:51:54.422 INFO:teuthology.orchestra.run.smithi079.stdout: } 2024-04-26T19:51:54.422 INFO:teuthology.orchestra.run.smithi079.stdout:] 2024-04-26T19:51:54.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e1c06c8f0 msgr2=0x7f2e1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:54.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e1c06c8f0 0x7f2e1c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2e20006fd0 tx=0x7f2e20008040 comp rx=0 tx=0).stop 2024-04-26T19:51:54.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e3006ff00 msgr2=0x7f2e3015f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:54.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e3006ff00 0x7f2e3015f040 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2e2400c7f0 tx=0x7f2e2400cbf0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 shutdown_connections 2024-04-26T19:51:54.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f2e1c06c8f0 0x7f2e1c06edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2e301488c0 0x7f2e301451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2e3006ff00 0x7f2e3015f040 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 --2- 172.21.15.79:0/3641571951 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2e3006fb20 0x7f2e301f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 >> 172.21.15.79:0/3641571951 conn(0x7f2e30071c30 msgr2=0x7f2e3006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:54.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 shutdown_connections 2024-04-26T19:51:54.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.423+0000 7f2e3573f640 1 -- 172.21.15.79:0/3641571951 wait complete. 2024-04-26T19:51:54.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2024-04-26T19:51:54.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.533+0000 7fa99afac640 1 Processor -- start 2024-04-26T19:51:54.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.533+0000 7fa99afac640 1 -- start start 2024-04-26T19:51:54.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa99afac640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9940afc30 0x7fa9940b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa99afac640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9940b3170 0x7fa994071850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa99afac640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa994071d90 0x7fa9940ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa99afac640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fa994072200 con 0x7fa9940afc30 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa99afac640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa9941640f0 con 0x7fa994071d90 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa99afac640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa9940b1eb0 con 0x7fa9940b3170 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa998d48640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9940afc30 0x7fa9940b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa998d48640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa9940afc30 0x7fa9940b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49218/0 (socket says 172.21.15.79:49218) 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa998d48640 1 -- 172.21.15.79:0/4050645043 learned_addr learned my addr 172.21.15.79:0/4050645043 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa993fff640 1 --2- 172.21.15.79:0/4050645043 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9940b3170 0x7fa994071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa999549640 1 --2- 172.21.15.79:0/4050645043 >> v2:172.21.15.83:3300/0 conn(0x7fa994071d90 0x7fa9940ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa998d48640 1 -- 172.21.15.79:0/4050645043 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9940b3170 msgr2=0x7fa994071850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa998d48640 1 --2- 172.21.15.79:0/4050645043 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9940b3170 0x7fa994071850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa998d48640 1 -- 172.21.15.79:0/4050645043 >> v2:172.21.15.83:3300/0 conn(0x7fa994071d90 msgr2=0x7fa9940ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa998d48640 1 --2- 172.21.15.79:0/4050645043 >> v2:172.21.15.83:3300/0 conn(0x7fa994071d90 0x7fa9940ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa998d48640 1 -- 172.21.15.79:0/4050645043 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9940b2150 con 0x7fa9940afc30 2024-04-26T19:51:54.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.534+0000 7fa999549640 1 --2- 172.21.15.79:0/4050645043 >> v2:172.21.15.83:3300/0 conn(0x7fa994071d90 0x7fa9940ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa998d48640 1 --2- 172.21.15.79:0/4050645043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940afc30 0x7fa9940b2c30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa9940702f0 tx=0x7fa98802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7e456fa60973bd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa9937fe640 1 -- 172.21.15.79:0/4050645043 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988037070 con 0x7fa9940afc30 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa9937fe640 1 -- 172.21.15.79:0/4050645043 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa988032440 con 0x7fa9940afc30 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa9937fe640 1 -- 172.21.15.79:0/4050645043 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa988032810 con 0x7fa9940afc30 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa99afac640 1 -- 172.21.15.79:0/4050645043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940afc30 msgr2=0x7fa9940b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa99afac640 1 --2- 172.21.15.79:0/4050645043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940afc30 0x7fa9940b2c30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa9940702f0 tx=0x7fa98802a140 comp rx=0 tx=0).stop 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa99afac640 1 -- 172.21.15.79:0/4050645043 shutdown_connections 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa99afac640 1 --2- 172.21.15.79:0/4050645043 >> v2:172.21.15.83:3300/0 conn(0x7fa994071d90 0x7fa9940ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa99afac640 1 --2- 172.21.15.79:0/4050645043 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa9940b3170 0x7fa994071850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa99afac640 1 --2- 172.21.15.79:0/4050645043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9940afc30 0x7fa9940b2c30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.535+0000 7fa99afac640 1 -- 172.21.15.79:0/4050645043 >> 172.21.15.79:0/4050645043 conn(0x7fa9940685d0 msgr2=0x7fa99406c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.536+0000 7fa99afac640 1 -- 172.21.15.79:0/4050645043 shutdown_connections 2024-04-26T19:51:54.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.536+0000 7fa99afac640 1 -- 172.21.15.79:0/4050645043 wait complete. 2024-04-26T19:51:54.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.537+0000 7fa99afac640 1 Processor -- start 2024-04-26T19:51:54.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.537+0000 7fa99afac640 1 -- start start 2024-04-26T19:51:54.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.537+0000 7fa99afac640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa994071d90 0x7fa9941e99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.537+0000 7fa99afac640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9940afc30 0x7fa994144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.541 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.537+0000 7fa99afac640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9941450d0 0x7fa994145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.537+0000 7fa99afac640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa9940b0010 con 0x7fa9941450d0 2024-04-26T19:51:54.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.537+0000 7fa99afac640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa9940b2710 con 0x7fa9940afc30 2024-04-26T19:51:54.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.537+0000 7fa99afac640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa9940b0190 con 0x7fa994071d90 2024-04-26T19:51:54.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa999549640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9941450d0 0x7fa994145810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa999549640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9941450d0 0x7fa994145810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49224/0 (socket says 172.21.15.79:49224) 2024-04-26T19:51:54.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa999549640 1 -- 172.21.15.79:0/73593209 learned_addr learned my addr 172.21.15.79:0/73593209 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:54.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa998d48640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa994071d90 0x7fa9941e99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa993fff640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9940afc30 0x7fa994144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa999549640 1 -- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa994071d90 msgr2=0x7fa9941e99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:54.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa999549640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa994071d90 0x7fa9941e99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa999549640 1 -- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9940afc30 msgr2=0x7fa994144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:54.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa999549640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9940afc30 0x7fa994144b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:54.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa999549640 1 -- 172.21.15.79:0/73593209 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9941e9fc0 con 0x7fa9941450d0 2024-04-26T19:51:54.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa993fff640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9940afc30 0x7fa994144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:54.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa998d48640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa994071d90 0x7fa9941e99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:51:54.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa999549640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9941450d0 0x7fa994145810 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00cef0 tx=0x7fa98c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:54.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa9917fa640 1 -- 172.21.15.79:0/73593209 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c01a070 con 0x7fa9941450d0 2024-04-26T19:51:54.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa9917fa640 1 -- 172.21.15.79:0/73593209 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa98c00d840 con 0x7fa9941450d0 2024-04-26T19:51:54.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa9917fa640 1 -- 172.21.15.79:0/73593209 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00dc30 con 0x7fa9941450d0 2024-04-26T19:51:54.544 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9941ea2a0 con 0x7fa9941450d0 2024-04-26T19:51:54.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.538+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9941ea860 con 0x7fa9941450d0 2024-04-26T19:51:54.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.539+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9940506c0 con 0x7fa9941450d0 2024-04-26T19:51:54.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.540+0000 7fa9917fa640 1 -- 172.21.15.79:0/73593209 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa98c01d070 con 0x7fa9941450d0 2024-04-26T19:51:54.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.540+0000 7fa9917fa640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa97806c8f0 0x7fa97806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:54.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.540+0000 7fa9917fa640 1 -- 172.21.15.79:0/73593209 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa98c08c500 con 0x7fa9941450d0 2024-04-26T19:51:54.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.540+0000 7fa998d48640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa97806c8f0 0x7fa97806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:54.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.541+0000 7fa998d48640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa97806c8f0 0x7fa97806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa988034000 tx=0x7fa98802e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:54.546 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.544+0000 7fa9917fa640 1 -- 172.21.15.79:0/73593209 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa98c05a830 con 0x7fa9941450d0 2024-04-26T19:51:54.768 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:54.766+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fa9940b1b60 con 0x7fa9941450d0 2024-04-26T19:51:55.110 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.109+0000 7fa9917fa640 1 -- 172.21.15.79:0/73593209 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) ==== 92+0+0 (secure 0 0 0) 0x7fa98c05a130 con 0x7fa9941450d0 2024-04-26T19:51:55.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.113+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa97806c8f0 msgr2=0x7fa97806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.113+0000 7fa99afac640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa97806c8f0 0x7fa97806edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa988034000 tx=0x7fa98802e000 comp rx=0 tx=0).stop 2024-04-26T19:51:55.114 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.113+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9941450d0 msgr2=0x7fa994145810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.113+0000 7fa99afac640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9941450d0 0x7fa994145810 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00cef0 tx=0x7fa98c00a710 comp rx=0 tx=0).stop 2024-04-26T19:51:55.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.114+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 shutdown_connections 2024-04-26T19:51:55.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.114+0000 7fa99afac640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fa97806c8f0 0x7fa97806edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.114+0000 7fa99afac640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa9941450d0 0x7fa994145810 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.114+0000 7fa99afac640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa9940afc30 0x7fa994144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.114+0000 7fa99afac640 1 --2- 172.21.15.79:0/73593209 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa994071d90 0x7fa9941e99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.114+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 >> 172.21.15.79:0/73593209 conn(0x7fa9940685d0 msgr2=0x7fa99406c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:55.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.114+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 shutdown_connections 2024-04-26T19:51:55.115 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.114+0000 7fa99afac640 1 -- 172.21.15.79:0/73593209 wait complete. 2024-04-26T19:51:55.131 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-04-26T19:51:55.228 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:55.226+0000 7f0b7fe12040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:51:55.233 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:55.232+0000 7fb9a1a19040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:51:55.234 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.234+0000 7fbb6f2fc640 1 Processor -- start 2024-04-26T19:51:55.235 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.234+0000 7fbb6f2fc640 1 -- start start 2024-04-26T19:51:55.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6f2fc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb68142480 0x7fbb68163010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6f2fc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 0x7fbb68144b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6f2fc640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb68163e90 con 0x7fbb681450a0 2024-04-26T19:51:55.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6f2fc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbb681635f0 con 0x7fbb68142480 2024-04-26T19:51:55.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6f2fc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbb681638d0 con 0x7fbb68071f90 2024-04-26T19:51:55.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6d098640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb68142480 0x7fbb68163010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.236 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6c897640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 0x7fbb68144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6d098640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbb68142480 0x7fbb68163010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:41698/0 (socket says 172.21.15.79:41698) 2024-04-26T19:51:55.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6d098640 1 -- 172.21.15.79:0/3377493599 learned_addr learned my addr 172.21.15.79:0/3377493599 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:55.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6d899640 1 --1- 172.21.15.79:0/3377493599 >> v1:172.21.15.83:6789/0 conn(0x7fbb681450a0 0x7fbb68160050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:36694/0 (socket says 172.21.15.79:36694) 2024-04-26T19:51:55.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6d098640 1 -- 172.21.15.79:0/3377493599 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 msgr2=0x7fbb68144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:55.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6d098640 1 --2- 172.21.15.79:0/3377493599 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 0x7fbb68144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6d098640 1 -- 172.21.15.79:0/3377493599 >> v1:172.21.15.83:6789/0 conn(0x7fbb681450a0 legacy=0x7fbb68160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.237 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.235+0000 7fbb6d098640 1 -- 172.21.15.79:0/3377493599 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb68163bb0 con 0x7fbb68142480 2024-04-26T19:51:55.238 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.236+0000 7fbb6d098640 1 --2- 172.21.15.79:0/3377493599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb68142480 0x7fbb68163010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009870 tx=0x7fbb5802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=855165de30aa6077 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.236+0000 7fbb5ffff640 1 -- 172.21.15.79:0/3377493599 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58037070 con 0x7fbb68142480 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.236+0000 7fbb5ffff640 1 -- 172.21.15.79:0/3377493599 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb58032440 con 0x7fbb68142480 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.236+0000 7fbb5ffff640 1 -- 172.21.15.79:0/3377493599 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58032810 con 0x7fbb68142480 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.237+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/3377493599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb68142480 msgr2=0x7fbb68163010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.237+0000 7fbb6f2fc640 1 --2- 172.21.15.79:0/3377493599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb68142480 0x7fbb68163010 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009870 tx=0x7fbb5802a270 comp rx=0 tx=0).stop 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.237+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/3377493599 shutdown_connections 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.237+0000 7fbb6f2fc640 1 --2- 172.21.15.79:0/3377493599 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 0x7fbb68144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.237+0000 7fbb6f2fc640 1 --2- 172.21.15.79:0/3377493599 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb68142480 0x7fbb68163010 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.237+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/3377493599 >> 172.21.15.79:0/3377493599 conn(0x7fbb680b3ed0 msgr2=0x7fbb680681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.237+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/3377493599 shutdown_connections 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.237+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/3377493599 wait complete. 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.238+0000 7fbb6f2fc640 1 Processor -- start 2024-04-26T19:51:55.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.238+0000 7fbb6f2fc640 1 -- start start 2024-04-26T19:51:55.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6f2fc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 0x7fbb681596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6f2fc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb68142480 0x7fbb68140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6f2fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb681450a0 0x7fbb68143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6f2fc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681608e0 con 0x7fbb681450a0 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6f2fc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68160760 con 0x7fbb68142480 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6f2fc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbb680d1610 con 0x7fbb68071f90 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6d899640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb681450a0 0x7fbb68143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6d899640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb681450a0 0x7fbb68143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49252/0 (socket says 172.21.15.79:49252) 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6d098640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 0x7fbb681596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6d899640 1 -- 172.21.15.79:0/2978869513 learned_addr learned my addr 172.21.15.79:0/2978869513 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:55.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6c897640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb68142480 0x7fbb68140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6d899640 1 -- 172.21.15.79:0/2978869513 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 msgr2=0x7fbb681596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6d899640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 0x7fbb681596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6d899640 1 -- 172.21.15.79:0/2978869513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb68142480 msgr2=0x7fbb68140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6d899640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb68142480 0x7fbb68140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.239+0000 7fbb6d899640 1 -- 172.21.15.79:0/2978869513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb681f9730 con 0x7fbb681450a0 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.240+0000 7fbb6d899640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb681450a0 0x7fbb68143fc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000c730 tx=0x7fbb6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.240+0000 7fbb5dffb640 1 -- 172.21.15.79:0/2978869513 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60017070 con 0x7fbb681450a0 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.240+0000 7fbb5dffb640 1 -- 172.21.15.79:0/2978869513 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb6000ce40 con 0x7fbb681450a0 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.240+0000 7fbb5dffb640 1 -- 172.21.15.79:0/2978869513 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb6000a840 con 0x7fbb681450a0 2024-04-26T19:51:55.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.240+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb681e9080 con 0x7fbb681450a0 2024-04-26T19:51:55.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.240+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb681e9640 con 0x7fbb681450a0 2024-04-26T19:51:55.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.241+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb680506c0 con 0x7fbb681450a0 2024-04-26T19:51:55.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.241+0000 7fbb5dffb640 1 -- 172.21.15.79:0/2978869513 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 6) ==== 91057+0+0 (secure 0 0 0) 0x7fbb6001a070 con 0x7fbb681450a0 2024-04-26T19:51:55.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.242+0000 7fbb5dffb640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb4c06c940 0x7fbb4c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.242+0000 7fbb6d098640 1 -- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb4c06c940 msgr2=0x7fbb4c06ee00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3161939478 2024-04-26T19:51:55.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.242+0000 7fbb5dffb640 1 -- 172.21.15.79:0/2978869513 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fbb6008b1e0 con 0x7fbb681450a0 2024-04-26T19:51:55.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.242+0000 7fbb6d098640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb4c06c940 0x7fbb4c06ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:51:55.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.246+0000 7fbb5dffb640 1 -- 172.21.15.79:0/2978869513 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb60058e30 con 0x7fbb681450a0 2024-04-26T19:51:55.340 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:55.339+0000 7f0b7fe12040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:51:55.351 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:55.350+0000 7fb9a1a19040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:51:55.386 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:55.385+0000 7f0b7fe12040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:51:55.399 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:55.398+0000 7fb9a1a19040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:51:55.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.430+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fbb68162d80 con 0x7fbb681450a0 2024-04-26T19:51:55.433 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.433+0000 7fbb5dffb640 1 -- 172.21.15.79:0/2978869513 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) ==== 138+0+0 (secure 0 0 0) 0x7fbb6005d130 con 0x7fbb681450a0 2024-04-26T19:51:55.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb4c06c940 msgr2=0x7fbb4c06ee00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:51:55.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb4c06c940 0x7fbb4c06ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb681450a0 msgr2=0x7fbb68143fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb681450a0 0x7fbb68143fc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbb6000c730 tx=0x7fbb6000cbf0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 shutdown_connections 2024-04-26T19:51:55.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7fbb4c06c940 0x7fbb4c06ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.436 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb681450a0 0x7fbb68143fc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb68142480 0x7fbb68140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 --2- 172.21.15.79:0/2978869513 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb68071f90 0x7fbb681596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 >> 172.21.15.79:0/2978869513 conn(0x7fbb680b3ed0 msgr2=0x7fbb68162070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:55.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 shutdown_connections 2024-04-26T19:51:55.437 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.435+0000 7fbb6f2fc640 1 -- 172.21.15.79:0/2978869513 wait complete. 2024-04-26T19:51:55.447 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-04-26T19:51:55.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.548+0000 7f57e71af640 1 Processor -- start 2024-04-26T19:51:55.548 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.548+0000 7f57e71af640 1 -- start start 2024-04-26T19:51:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57e71af640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f57e00aeeb0 0x7f57e006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57e71af640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f57e0070160 0x7f57e00b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57e71af640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00b1990 0x7f57e00af290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57e71af640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f57e006fa10 con 0x7f57e0070160 2024-04-26T19:51:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57e71af640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f57e0072ea0 con 0x7f57e00aeeb0 2024-04-26T19:51:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57e71af640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f57e0073180 con 0x7f57e00b1990 2024-04-26T19:51:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57dffff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f57e0070160 0x7f57e00b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57dffff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f57e0070160 0x7f57e00b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49258/0 (socket says 172.21.15.79:49258) 2024-04-26T19:51:55.551 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57dffff640 1 -- 172.21.15.79:0/4243108592 learned_addr learned my addr 172.21.15.79:0/4243108592 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57e4f4b640 1 --2- 172.21.15.79:0/4243108592 >> v2:172.21.15.83:3300/0 conn(0x7f57e00aeeb0 0x7f57e006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57e574c640 1 --2- 172.21.15.79:0/4243108592 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00b1990 0x7f57e00af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57dffff640 1 -- 172.21.15.79:0/4243108592 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00b1990 msgr2=0x7f57e00af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57dffff640 1 --2- 172.21.15.79:0/4243108592 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00b1990 0x7f57e00af290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57dffff640 1 -- 172.21.15.79:0/4243108592 >> v2:172.21.15.83:3300/0 conn(0x7f57e00aeeb0 msgr2=0x7f57e006f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57dffff640 1 --2- 172.21.15.79:0/4243108592 >> v2:172.21.15.83:3300/0 conn(0x7f57e00aeeb0 0x7f57e006f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.549+0000 7f57dffff640 1 -- 172.21.15.79:0/4243108592 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e0073460 con 0x7f57e0070160 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.550+0000 7f57dffff640 1 --2- 172.21.15.79:0/4243108592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e0070160 0x7f57e00b1450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f57e00728c0 tx=0x7f57cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7c341514907bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.550+0000 7f57df7fe640 1 -- 172.21.15.79:0/4243108592 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc037070 con 0x7f57e0070160 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.550+0000 7f57df7fe640 1 -- 172.21.15.79:0/4243108592 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57cc032440 con 0x7f57e0070160 2024-04-26T19:51:55.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.550+0000 7f57df7fe640 1 -- 172.21.15.79:0/4243108592 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc032810 con 0x7f57e0070160 2024-04-26T19:51:55.553 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.551+0000 7f57e71af640 1 -- 172.21.15.79:0/4243108592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e0070160 msgr2=0x7f57e00b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.551+0000 7f57e71af640 1 --2- 172.21.15.79:0/4243108592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e0070160 0x7f57e00b1450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f57e00728c0 tx=0x7f57cc02a140 comp rx=0 tx=0).stop 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.551+0000 7f57e71af640 1 -- 172.21.15.79:0/4243108592 shutdown_connections 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.551+0000 7f57e71af640 1 --2- 172.21.15.79:0/4243108592 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00b1990 0x7f57e00af290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.551+0000 7f57e71af640 1 --2- 172.21.15.79:0/4243108592 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e0070160 0x7f57e00b1450 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.551+0000 7f57e71af640 1 --2- 172.21.15.79:0/4243108592 >> v2:172.21.15.83:3300/0 conn(0x7f57e00aeeb0 0x7f57e006f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.551+0000 7f57e71af640 1 -- 172.21.15.79:0/4243108592 >> 172.21.15.79:0/4243108592 conn(0x7f57e00687e0 msgr2=0x7f57e0071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.551+0000 7f57e71af640 1 -- 172.21.15.79:0/4243108592 shutdown_connections 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.551+0000 7f57e71af640 1 -- 172.21.15.79:0/4243108592 wait complete. 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.553+0000 7f57e71af640 1 Processor -- start 2024-04-26T19:51:55.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.553+0000 7f57e71af640 1 -- start start 2024-04-26T19:51:55.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.554+0000 7f57e71af640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e0070160 0x7f57e00d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.554+0000 7f57e71af640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00aeeb0 0x7f57e01623b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.554+0000 7f57e71af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e00b1990 0x7f57e013bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.554+0000 7f57e71af640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f57e006f620 con 0x7f57e00b1990 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.554+0000 7f57e71af640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f57e0149380 con 0x7f57e0070160 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e71af640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f57e006f7a0 con 0x7f57e00aeeb0 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e574c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e00b1990 0x7f57e013bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e574c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e00b1990 0x7f57e013bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49272/0 (socket says 172.21.15.79:49272) 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e574c640 1 -- 172.21.15.79:0/1868973079 learned_addr learned my addr 172.21.15.79:0/1868973079 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e4f4b640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e0070160 0x7f57e00d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57dffff640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00aeeb0 0x7f57e01623b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e574c640 1 -- 172.21.15.79:0/1868973079 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00aeeb0 msgr2=0x7f57e01623b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e574c640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00aeeb0 0x7f57e01623b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e574c640 1 -- 172.21.15.79:0/1868973079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e0070160 msgr2=0x7f57e00d3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.557 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e574c640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e0070160 0x7f57e00d3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e574c640 1 -- 172.21.15.79:0/1868973079 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e01f8770 con 0x7f57e00b1990 2024-04-26T19:51:55.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e574c640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e00b1990 0x7f57e013bb10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f57d8007e40 tx=0x7f57d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:55.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57dd7fa640 1 -- 172.21.15.79:0/1868973079 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8014620 con 0x7f57e00b1990 2024-04-26T19:51:55.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57dd7fa640 1 -- 172.21.15.79:0/1868973079 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d8019070 con 0x7f57e00b1990 2024-04-26T19:51:55.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e01f8a50 con 0x7f57e00b1990 2024-04-26T19:51:55.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57dd7fa640 1 -- 172.21.15.79:0/1868973079 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8014a20 con 0x7f57e00b1990 2024-04-26T19:51:55.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.555+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e01f9010 con 0x7f57e00b1990 2024-04-26T19:51:55.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.556+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57e00506c0 con 0x7f57e00b1990 2024-04-26T19:51:55.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.557+0000 7f57dd7fa640 1 -- 172.21.15.79:0/1868973079 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 6) ==== 91057+0+0 (secure 0 0 0) 0x7f57d8014bc0 con 0x7f57e00b1990 2024-04-26T19:51:55.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.557+0000 7f57dd7fa640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f57c406c940 0x7f57c406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.557+0000 7f57e4f4b640 1 -- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f57c406c940 msgr2=0x7f57c406ee00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3161939478 2024-04-26T19:51:55.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.557+0000 7f57dd7fa640 1 -- 172.21.15.79:0/1868973079 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f57d808c4c0 con 0x7f57e00b1990 2024-04-26T19:51:55.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.557+0000 7f57e4f4b640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f57c406c940 0x7f57c406ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:51:55.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.562+0000 7f57dd7fa640 1 -- 172.21.15.79:0/1868973079 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f57d805a110 con 0x7f57e00b1990 2024-04-26T19:51:55.589 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:55.588+0000 7f0b7fe12040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:51:55.619 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:55.616+0000 7fb9a1a19040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:51:55.631 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:55.630+0000 7f0b7fe12040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:51:55.661 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:55.661+0000 7fb9a1a19040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:51:55.752 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.745+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f57e006a800 con 0x7f57e00b1990 2024-04-26T19:51:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.748+0000 7f57dd7fa640 1 -- 172.21.15.79:0/1868973079 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) ==== 133+0+0 (secure 0 0 0) 0x7f57d805e410 con 0x7f57e00b1990 2024-04-26T19:51:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f57c406c940 msgr2=0x7f57c406ee00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:51:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f57c406c940 0x7f57c406ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.753 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e00b1990 msgr2=0x7f57e013bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e00b1990 0x7f57e013bb10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f57d8007e40 tx=0x7f57d800e4b0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 shutdown_connections 2024-04-26T19:51:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f57c406c940 0x7f57c406ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f57e00b1990 0x7f57e013bb10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f57e00aeeb0 0x7f57e01623b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.754 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 --2- 172.21.15.79:0/1868973079 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f57e0070160 0x7f57e00d3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 >> 172.21.15.79:0/1868973079 conn(0x7f57e00687e0 msgr2=0x7f57e00b1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 shutdown_connections 2024-04-26T19:51:55.755 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.750+0000 7f57e71af640 1 -- 172.21.15.79:0/1868973079 wait complete. 2024-04-26T19:51:55.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:51:55.764 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2024-04-26T19:51:55.784 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1m 2024-04-26T19:51:55.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.924+0000 7f429b59e640 1 Processor -- start 2024-04-26T19:51:55.924 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.924+0000 7f429b59e640 1 -- start start 2024-04-26T19:51:55.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f429b59e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f429406d790 0x7f429406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f429b59e640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4294163370 con 0x7f4294162b30 2024-04-26T19:51:55.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f429b59e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4294163650 con 0x7f42940b3b90 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f429b59e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4294162f10 con 0x7f429406d790 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f429933a640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4294162b30 0x7f42940b3480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:53660/0 (socket says 172.21.15.79:53660) 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f429933a640 1 -- 172.21.15.79:0/895596368 learned_addr learned my addr 172.21.15.79:0/895596368 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f4299b3b640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f429406d790 0x7f429406ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1186244176 0 0) 0x7f4294163370 con 0x7f4294162b30 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4280003520 con 0x7f4294162b30 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2947762330 0 0) 0x7f4294163650 con 0x7f42940b3b90 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4294163370 con 0x7f42940b3b90 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 645116181 0 0) 0x7f4280003520 con 0x7f4294162b30 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f429406d790 msgr2=0x7f429406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f429406d790 0x7f429406ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 >> v1:172.21.15.83:6789/0 conn(0x7f42940b3b90 legacy=0x7f429406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4294163930 con 0x7f4294162b30 2024-04-26T19:51:55.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f4290001670 con 0x7f4294162b30 2024-04-26T19:51:55.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 >> v1:172.21.15.79:6789/0 conn(0x7f4294162b30 legacy=0x7f42940b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4280003b70 0x7f4280003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4280004800 0x7f4280004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f42800051c0 0x7f428000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4294162f10 con 0x7f4280004800 2024-04-26T19:51:55.929 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f428000c030 con 0x7f4280003b70 2024-04-26T19:51:55.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f428000c230 con 0x7f42800051c0 2024-04-26T19:51:55.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.925+0000 7f4299b3b640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f429406d790 0x7f429406ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:55.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.926+0000 7f429933a640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4280003b70 0x7f4280003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.926+0000 7f4298b39640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4280004800 0x7f4280004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.926+0000 7f4299b3b640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f42800051c0 0x7f428000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.930 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.926+0000 7f429933a640 1 -- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f42800051c0 msgr2=0x7f428000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:55.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.926+0000 7f429933a640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f42800051c0 0x7f428000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.926+0000 7f429933a640 1 -- 172.21.15.79:0/895596368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4280004800 msgr2=0x7f4280004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:55.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.926+0000 7f429933a640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4280004800 0x7f4280004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.926+0000 7f429933a640 1 -- 172.21.15.79:0/895596368 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4280003520 con 0x7f4280003b70 2024-04-26T19:51:55.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.926+0000 7f4298b39640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4280004800 0x7f4280004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:55.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.927+0000 7f429933a640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4280003b70 0x7f4280003f70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f429000c1c0 tx=0x7f4280005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=109c290836234700 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:55.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.927+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290007cc0 con 0x7f4280003b70 2024-04-26T19:51:55.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.927+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4290005af0 con 0x7f4280003b70 2024-04-26T19:51:55.932 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.927+0000 7f428bfff640 1 -- 172.21.15.79:0/895596368 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290007070 con 0x7f4280003b70 2024-04-26T19:51:55.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.928+0000 7f429b59e640 1 -- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4280003b70 msgr2=0x7f4280003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.928+0000 7f429b59e640 1 --2- 172.21.15.79:0/895596368 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4280003b70 0x7f4280003f70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f429000c1c0 tx=0x7f4280005830 comp rx=0 tx=0).stop 2024-04-26T19:51:55.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.928+0000 7f4299b3b640 1 -- 172.21.15.79:0/895596368 reap_dead start 2024-04-26T19:51:55.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.928+0000 7f429b59e640 1 -- 172.21.15.79:0/895596368 shutdown_connections 2024-04-26T19:51:55.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.928+0000 7f429b59e640 1 -- 172.21.15.79:0/895596368 >> 172.21.15.79:0/895596368 conn(0x7f42940b4500 msgr2=0x7f42940b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:55.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.928+0000 7f429b59e640 1 -- 172.21.15.79:0/895596368 shutdown_connections 2024-04-26T19:51:55.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.928+0000 7f429b59e640 1 -- 172.21.15.79:0/895596368 wait complete. 2024-04-26T19:51:55.936 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.929+0000 7f429b59e640 1 Processor -- start 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.930+0000 7f429b59e640 1 -- start start 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f429b59e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4280003b70 0x7f42940cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f429b59e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294156300 0x7f4294157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f429b59e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429406f920 0x7f429406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f429b59e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f42940d02f0 con 0x7f429406f920 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f429b59e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4294163120 con 0x7f4294156300 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f429b59e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f42940b1900 con 0x7f4280003b70 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4299b3b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429406f920 0x7f429406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4299b3b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429406f920 0x7f429406fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:49300/0 (socket says 172.21.15.79:49300) 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4299b3b640 1 -- 172.21.15.79:0/4101917731 learned_addr learned my addr 172.21.15.79:0/4101917731 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f429933a640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4280003b70 0x7f42940cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4298b39640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294156300 0x7f4294157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4299b3b640 1 -- 172.21.15.79:0/4101917731 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4280003b70 msgr2=0x7f42940cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.937 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4299b3b640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4280003b70 0x7f42940cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4299b3b640 1 -- 172.21.15.79:0/4101917731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294156300 msgr2=0x7f4294157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4299b3b640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4294156300 0x7f4294157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4299b3b640 1 -- 172.21.15.79:0/4101917731 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42941f4a80 con 0x7f429406f920 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f429933a640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4280003b70 0x7f42940cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4299b3b640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429406f920 0x7f429406fda0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f429406db70 tx=0x7f428c00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c01e070 con 0x7f429406f920 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f428c003990 con 0x7f429406f920 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.931+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42940cccb0 con 0x7f429406f920 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.932+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42840014e0 con 0x7f429406f920 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.932+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42940cd270 con 0x7f429406f920 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.933+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42940702e0 con 0x7f429406f920 2024-04-26T19:51:55.938 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.933+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 6) ==== 91057+0+0 (secure 0 0 0) 0x7f428c021070 con 0x7f429406f920 2024-04-26T19:51:55.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.934+0000 7f4289ffb640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 0x7f427806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:55.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.934+0000 7f429933a640 1 -- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 msgr2=0x7f427806ee00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3161939478 2024-04-26T19:51:55.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.934+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f428c08bbb0 con 0x7f429406f920 2024-04-26T19:51:55.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.934+0000 7f429933a640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 0x7f427806ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:51:55.939 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:55.938+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f428c010040 con 0x7f429406f920 2024-04-26T19:51:56.114 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:56.113+0000 7f0b7fe12040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:51:56.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:56.135+0000 7f429933a640 1 -- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 msgr2=0x7f427806ee00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3161939478 2024-04-26T19:51:56.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:56.135+0000 7f429933a640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 0x7f427806ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T19:51:56.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:56.142+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 --> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f42940506c0 con 0x7f427806c940 2024-04-26T19:51:56.168 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:56.167+0000 7fb9a1a19040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:51:56.198 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:56.197+0000 7f0b7fe12040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:51:56.240 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:56.239+0000 7f0b7fe12040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:51:56.251 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:56.251+0000 7fb9a1a19040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:51:56.292 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:56.292+0000 7fb9a1a19040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:51:56.401 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:56.399+0000 7f0b7fe12040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:51:56.446 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:56.443+0000 7f0b7fe12040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:51:56.452 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:56.452+0000 7fb9a1a19040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:51:56.493 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:56.492+0000 7f0b7fe12040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:51:56.496 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:56.496+0000 7fb9a1a19040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:51:56.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:56.536+0000 7f429933a640 1 -- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 msgr2=0x7f427806ee00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3161939478 2024-04-26T19:51:56.536 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:56.536+0000 7f429933a640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 0x7f427806ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T19:51:56.545 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:56.544+0000 7fb9a1a19040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:51:56.603 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:56.602+0000 7f0b7fe12040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:51:56.654 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:56.653+0000 7fb9a1a19040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:51:56.915 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:56.914+0000 7f0b7fe12040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:51:56.969 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:56.968+0000 7fb9a1a19040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:51:57.124 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:57.123+0000 7f0b7fe12040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:51:57.173 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:57.172+0000 7f0b7fe12040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:51:57.178 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:57.178+0000 7fb9a1a19040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:51:57.227 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:57.227+0000 7fb9a1a19040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:51:57.286 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:57.285+0000 7f0b7fe12040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:51:57.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:57.337+0000 7f429933a640 1 -- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 msgr2=0x7f427806ee00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3161939478 2024-04-26T19:51:57.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:57.337+0000 7f429933a640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 0x7f427806ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T19:51:57.341 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:57.341+0000 7fb9a1a19040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:51:57.552 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:57.551+0000 7f0b7fe12040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:51:57.595 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:57.594+0000 7f0b7fe12040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:51:57.611 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:57.610+0000 7fb9a1a19040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:51:57.653 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:57.653+0000 7fb9a1a19040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:51:57.737 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:57.736+0000 7f0b7fe12040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:51:57.796 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:57.796+0000 7fb9a1a19040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:51:57.867 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:57.865+0000 7f0b7fe12040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:51:57.915 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:57.913+0000 7f0b7fe12040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:51:57.928 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:57.927+0000 7fb9a1a19040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:51:57.968 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:57.967+0000 7f0b7fe12040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:51:57.975 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:57.975+0000 7fb9a1a19040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:51:58.010 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:58.009+0000 7f0b7fe12040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:51:58.029 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.028+0000 7fb9a1a19040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:51:58.070 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.070+0000 7fb9a1a19040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:51:58.397 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:58.396+0000 7f0b7fe12040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:51:58.458 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.457+0000 7fb9a1a19040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:51:58.512 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:51:58.510+0000 7f0b7fe12040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:51:58.570 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:58.569+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 7) ==== 91057+0+0 (secure 0 0 0) 0x7f428c055e70 con 0x7f429406f920 2024-04-26T19:51:58.574 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.573+0000 7fb9a1a19040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:51:58.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:58.583+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mon.0 v2:172.21.15.79:3300/0 8 ==== mgrmap(e 8) ==== 90768+0+0 (secure 0 0 0) 0x7f428c060df0 con 0x7f429406f920 2024-04-26T19:51:58.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:58.583+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 msgr2=0x7f427806ee00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:51:58.584 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:58.583+0000 7f4289ffb640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3161939478,v1:172.21.15.79:6833/3161939478] conn(0x7f427806c940 0x7f427806ee00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91ffd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91ffd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91ffd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91ffd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91ffd0640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91cfca640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91cfca640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91cfca640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91cfca640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:58.741 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:51:58.740+0000 7fb91cfca640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:51:59.586 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.585+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mon.0 v2:172.21.15.79:3300/0 9 ==== mgrmap(e 9) ==== 90895+0+0 (secure 0 0 0) 0x7f428c056740 con 0x7f429406f920 2024-04-26T19:51:59.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.586+0000 7f4289ffb640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4278028750 0x7f4278076e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:59.587 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.586+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 --> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f428c060b20 con 0x7f4278028750 2024-04-26T19:51:59.589 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.588+0000 7f429933a640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4278028750 0x7f4278076e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:59.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.591+0000 7f429933a640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4278028750 0x7f4278076e50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4290032000 tx=0x7f4290009030 comp rx=0 tx=0).ready entity=mgr.5580 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:59.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.677+0000 7f4289ffb640 1 -- 172.21.15.79:0/4101917731 <== mgr.5580 v2:172.21.15.79:6832/3922824612 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f428c060b20 con 0x7f4278028750 2024-04-26T19:51:59.678 INFO:teuthology.orchestra.run.smithi079.stdout:Schedule set for path snap_test_dir1 2024-04-26T19:51:59.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.679+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4278028750 msgr2=0x7f4278076e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:59.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.679+0000 7f429b59e640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4278028750 0x7f4278076e50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4290032000 tx=0x7f4290009030 comp rx=0 tx=0).stop 2024-04-26T19:51:59.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.679+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429406f920 msgr2=0x7f429406fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:59.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.679+0000 7f429b59e640 1 --2- 172.21.15.79:0/4101917731 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f429406f920 0x7f429406fda0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f429406db70 tx=0x7f428c00f7b0 comp rx=0 tx=0).stop 2024-04-26T19:51:59.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.679+0000 7f4299b3b640 1 -- 172.21.15.79:0/4101917731 reap_dead start 2024-04-26T19:51:59.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.679+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 shutdown_connections 2024-04-26T19:51:59.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.679+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 >> 172.21.15.79:0/4101917731 conn(0x7f42940b4500 msgr2=0x7f4294158bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:59.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.680+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 shutdown_connections 2024-04-26T19:51:59.681 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.680+0000 7f429b59e640 1 -- 172.21.15.79:0/4101917731 wait complete. 2024-04-26T19:51:59.691 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2024-04-26T19:51:59.692 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1m 2024-04-26T19:51:59.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.793+0000 7f3b1cfe9640 1 Processor -- start 2024-04-26T19:51:59.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.793+0000 7f3b1cfe9640 1 -- start start 2024-04-26T19:51:59.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.794+0000 7f3b1cfe9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b180ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:59.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.794+0000 7f3b1cfe9640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b18072930 0x7f3b1806eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.794+0000 7f3b1cfe9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b180af370 con 0x7f3b18145750 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.794+0000 7f3b1cfe9640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3b180af650 con 0x7f3b18072930 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.794+0000 7f3b1cfe9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3b180af930 con 0x7f3b18145370 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.794+0000 7f3b1659c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b180ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.794+0000 7f3b16d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b18072930 0x7f3b1806eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.795+0000 7f3b1659c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b180ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:48312/0 (socket says 172.21.15.79:48312) 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.795+0000 7f3b16d9d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3b18072930 0x7f3b1806eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:54106/0 (socket says 172.21.15.79:54106) 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.795+0000 7f3b1659c640 1 -- 172.21.15.79:0/4002522811 learned_addr learned my addr 172.21.15.79:0/4002522811 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.795+0000 7f3b1659c640 1 -- 172.21.15.79:0/4002522811 >> v2:172.21.15.83:3300/0 conn(0x7f3b18072930 msgr2=0x7f3b1806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.795+0000 7f3b1659c640 1 --2- 172.21.15.79:0/4002522811 >> v2:172.21.15.83:3300/0 conn(0x7f3b18072930 0x7f3b1806eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:59.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.795+0000 7f3b1659c640 1 -- 172.21.15.79:0/4002522811 >> v1:172.21.15.83:6789/0 conn(0x7f3b18145750 legacy=0x7f3b18072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:59.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.795+0000 7f3b1659c640 1 -- 172.21.15.79:0/4002522811 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b180afc10 con 0x7f3b18145370 2024-04-26T19:51:59.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.795+0000 7f3b16d9d640 1 --2- 172.21.15.79:0/4002522811 >> v2:172.21.15.83:3300/0 conn(0x7f3b18072930 0x7f3b1806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:51:59.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1659c640 1 --2- 172.21.15.79:0/4002522811 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b180ae8d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8304973d38ba4091 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:59.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1559a640 1 -- 172.21.15.79:0/4002522811 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08036070 con 0x7f3b18145370 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1559a640 1 -- 172.21.15.79:0/4002522811 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b08031600 con 0x7f3b18145370 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1559a640 1 -- 172.21.15.79:0/4002522811 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b080319d0 con 0x7f3b18145370 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/4002522811 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 msgr2=0x7f3b180ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1cfe9640 1 --2- 172.21.15.79:0/4002522811 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b180ae8d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009e50 tx=0x7f3b0802a710 comp rx=0 tx=0).stop 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/4002522811 shutdown_connections 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1cfe9640 1 --2- 172.21.15.79:0/4002522811 >> v2:172.21.15.83:3300/0 conn(0x7f3b18072930 0x7f3b1806eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1cfe9640 1 --2- 172.21.15.79:0/4002522811 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b180ae8d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/4002522811 >> 172.21.15.79:0/4002522811 conn(0x7f3b180b31c0 msgr2=0x7f3b180b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/4002522811 shutdown_connections 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.796+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/4002522811 wait complete. 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.798+0000 7f3b1cfe9640 1 Processor -- start 2024-04-26T19:51:59.802 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.798+0000 7f3b1cfe9640 1 -- start start 2024-04-26T19:51:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.798+0000 7f3b1cfe9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b18072930 0x7f3b180ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.798+0000 7f3b1cfe9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b1813c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:59.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.798+0000 7f3b1cfe9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1813ca00 0x7f3b1806fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.798+0000 7f3b1cfe9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3b180aee10 con 0x7f3b18072930 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.798+0000 7f3b1cfe9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3b18071a80 con 0x7f3b1813ca00 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.798+0000 7f3b1cfe9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3b180aef90 con 0x7f3b18145370 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.798+0000 7f3b1659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b18072930 0x7f3b180ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1659c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b18072930 0x7f3b180ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54522/0 (socket says 172.21.15.79:54522) 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1659c640 1 -- 172.21.15.79:0/1428061917 learned_addr learned my addr 172.21.15.79:0/1428061917 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b16d9d640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1813ca00 0x7f3b1806fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b15d9b640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b1813c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1659c640 1 -- 172.21.15.79:0/1428061917 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 msgr2=0x7f3b1813c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1659c640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b1813c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1659c640 1 -- 172.21.15.79:0/1428061917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1813ca00 msgr2=0x7f3b1806fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:51:59.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1659c640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1813ca00 0x7f3b1806fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1659c640 1 -- 172.21.15.79:0/1428061917 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1813b700 con 0x7f3b18072930 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1659c640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b18072930 0x7f3b180ce9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009f70 tx=0x7f3b08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b06ffd640 1 -- 172.21.15.79:0/1428061917 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08036050 con 0x7f3b18072930 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b06ffd640 1 -- 172.21.15.79:0/1428061917 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b08042070 con 0x7f3b18072930 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1813b9e0 con 0x7f3b18072930 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b06ffd640 1 -- 172.21.15.79:0/1428061917 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b08009510 con 0x7f3b18072930 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.799+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b180cef10 con 0x7f3b18072930 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.800+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b180b5460 con 0x7f3b18072930 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.801+0000 7f3b06ffd640 1 -- 172.21.15.79:0/1428061917 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 9) ==== 90895+0+0 (secure 0 0 0) 0x7f3b08007c50 con 0x7f3b18072930 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.801+0000 7f3b06ffd640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f3af806c790 0x7f3af806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.801+0000 7f3b06ffd640 1 -- 172.21.15.79:0/1428061917 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f3b080ac890 con 0x7f3b18072930 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.801+0000 7f3b15d9b640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f3af806c790 0x7f3af806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:51:59.807 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.801+0000 7f3b15d9b640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f3af806c790 0x7f3af806ec50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3b1813d0a0 tx=0x7f3b0c009040 comp rx=0 tx=0).ready entity=mgr.5580 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:51:59.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:51:59.806+0000 7f3b06ffd640 1 -- 172.21.15.79:0/1428061917 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3b180b5460 con 0x7f3b18072930 2024-04-26T19:52:00.004 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.003+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 --> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1m", "target": ["mon-mgr", ""]}) -- 0x7f3b180cf300 con 0x7f3af806c790 2024-04-26T19:52:00.063 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.062+0000 7f3b06ffd640 1 -- 172.21.15.79:0/1428061917 <== mgr.5580 v2:172.21.15.79:6832/3922824612 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+39 (secure 0 0 0) 0x7f3b180cf300 con 0x7f3af806c790 2024-04-26T19:52:00.063 INFO:teuthology.orchestra.run.smithi079.stdout:Retention added to path snap_test_dir1 2024-04-26T19:52:00.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.064+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f3af806c790 msgr2=0x7f3af806ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.064+0000 7f3b1cfe9640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f3af806c790 0x7f3af806ec50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3b1813d0a0 tx=0x7f3b0c009040 comp rx=0 tx=0).stop 2024-04-26T19:52:00.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.064+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b18072930 msgr2=0x7f3b180ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.064+0000 7f3b1cfe9640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b18072930 0x7f3b180ce9d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009f70 tx=0x7f3b08004a40 comp rx=0 tx=0).stop 2024-04-26T19:52:00.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.065+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 shutdown_connections 2024-04-26T19:52:00.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.065+0000 7f3b1cfe9640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f3af806c790 0x7f3af806ec50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.065+0000 7f3b1cfe9640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3b1813ca00 0x7f3b1806fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.065+0000 7f3b1cfe9640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3b18145370 0x7f3b1813c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.065+0000 7f3b1cfe9640 1 --2- 172.21.15.79:0/1428061917 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3b18072930 0x7f3b180ce9d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.065+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 >> 172.21.15.79:0/1428061917 conn(0x7f3b180b31c0 msgr2=0x7f3b1806eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:00.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.065+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 shutdown_connections 2024-04-26T19:52:00.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.065+0000 7f3b1cfe9640 1 -- 172.21.15.79:0/1428061917 wait complete. 2024-04-26T19:52:00.076 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2024-04-26T19:52:00.077 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.customsnapkernel/scheduled-2024-04-26-19_53_00 in ~60s... 2024-04-26T19:52:00.077 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1m'], expected_retention: [{'m': 1}] 2024-04-26T19:52:00.077 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2024-04-26T19:52:00.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.175+0000 7fbb18ea0640 1 Processor -- start 2024-04-26T19:52:00.176 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.175+0000 7fbb18ea0640 1 -- start start 2024-04-26T19:52:00.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb18ea0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb140b3850 0x7fbb140b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb18ea0640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbb141605f0 0x7fbb14160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb18ea0640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb140b25b0 con 0x7fbb141564e0 2024-04-26T19:52:00.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb18ea0640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fbb140b2390 con 0x7fbb141605f0 2024-04-26T19:52:00.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb18ea0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbb140b41d0 con 0x7fbb140b3850 2024-04-26T19:52:00.177 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb12d9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbb141605f0 0x7fbb14160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb1259c640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbb141564e0 0x7fbb140b1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59712/0 (socket says 172.21.15.79:59712) 2024-04-26T19:52:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb1259c640 1 -- 172.21.15.79:0/3120122021 learned_addr learned my addr 172.21.15.79:0/3120122021 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb12d9d640 1 -- 172.21.15.79:0/3120122021 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb140b3850 msgr2=0x7fbb140b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb11d9b640 1 --2- 172.21.15.79:0/3120122021 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb140b3850 0x7fbb140b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb1159a640 1 -- 172.21.15.79:0/3120122021 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2396220 0 0) 0x7fbb140b25b0 con 0x7fbb141564e0 2024-04-26T19:52:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb12d9d640 1 --2- 172.21.15.79:0/3120122021 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb140b3850 0x7fbb140b3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb12d9d640 1 -- 172.21.15.79:0/3120122021 >> v1:172.21.15.79:6789/0 conn(0x7fbb141564e0 legacy=0x7fbb140b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.178 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.176+0000 7fbb12d9d640 1 -- 172.21.15.79:0/3120122021 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1406c3f0 con 0x7fbb141605f0 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.177+0000 7fbb12d9d640 1 --2- 172.21.15.79:0/3120122021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb141605f0 0x7fbb14160a60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009d90 tx=0x7fbb0c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725dcbdf0e71c904 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.177+0000 7fbb1159a640 1 -- 172.21.15.79:0/3120122021 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c007740 con 0x7fbb141605f0 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.177+0000 7fbb1159a640 1 -- 172.21.15.79:0/3120122021 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0c0078e0 con 0x7fbb141605f0 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.177+0000 7fbb1159a640 1 -- 172.21.15.79:0/3120122021 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c007cb0 con 0x7fbb141605f0 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.178+0000 7fbb18ea0640 1 -- 172.21.15.79:0/3120122021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb141605f0 msgr2=0x7fbb14160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.178+0000 7fbb18ea0640 1 --2- 172.21.15.79:0/3120122021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb141605f0 0x7fbb14160a60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009d90 tx=0x7fbb0c02a660 comp rx=0 tx=0).stop 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.178+0000 7fbb18ea0640 1 -- 172.21.15.79:0/3120122021 shutdown_connections 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.178+0000 7fbb18ea0640 1 --2- 172.21.15.79:0/3120122021 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb141605f0 0x7fbb14160a60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.178+0000 7fbb18ea0640 1 --2- 172.21.15.79:0/3120122021 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb140b3850 0x7fbb140b3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.178+0000 7fbb18ea0640 1 -- 172.21.15.79:0/3120122021 >> 172.21.15.79:0/3120122021 conn(0x7fbb14068180 msgr2=0x7fbb14068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.178+0000 7fbb18ea0640 1 -- 172.21.15.79:0/3120122021 shutdown_connections 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.178+0000 7fbb18ea0640 1 -- 172.21.15.79:0/3120122021 wait complete. 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.179+0000 7fbb18ea0640 1 Processor -- start 2024-04-26T19:52:00.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.179+0000 7fbb18ea0640 1 -- start start 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb18ea0640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb140b3850 0x7fbb14155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb18ea0640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb141564e0 0x7fbb1415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb18ea0640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb141605f0 0x7fbb141e5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb18ea0640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1406cc90 con 0x7fbb140b3850 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb18ea0640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1406c9b0 con 0x7fbb141605f0 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb18ea0640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbb1406d1b0 con 0x7fbb141564e0 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb140b3850 0x7fbb14155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb140b3850 0x7fbb14155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54530/0 (socket says 172.21.15.79:54530) 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb1259c640 1 -- 172.21.15.79:0/1404163720 learned_addr learned my addr 172.21.15.79:0/1404163720 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb11d9b640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb141564e0 0x7fbb1415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb12d9d640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb141605f0 0x7fbb141e5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb1259c640 1 -- 172.21.15.79:0/1404163720 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb141564e0 msgr2=0x7fbb1415b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb1259c640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb141564e0 0x7fbb1415b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb1259c640 1 -- 172.21.15.79:0/1404163720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb141605f0 msgr2=0x7fbb141e5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb1259c640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb141605f0 0x7fbb141e5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.180+0000 7fbb1259c640 1 -- 172.21.15.79:0/1404163720 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb141e61d0 con 0x7fbb140b3850 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.181+0000 7fbb1259c640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb140b3850 0x7fbb14155bb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400c500 tx=0x7fbb0400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.181+0000 7fbb02ffd640 1 -- 172.21.15.79:0/1404163720 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04018070 con 0x7fbb140b3850 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.181+0000 7fbb02ffd640 1 -- 172.21.15.79:0/1404163720 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb04007580 con 0x7fbb140b3850 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.181+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb141e64b0 con 0x7fbb140b3850 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.181+0000 7fbb02ffd640 1 -- 172.21.15.79:0/1404163720 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04004a20 con 0x7fbb140b3850 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.181+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1415dd40 con 0x7fbb140b3850 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.182+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb140506c0 con 0x7fbb140b3850 2024-04-26T19:52:00.184 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.182+0000 7fbb02ffd640 1 -- 172.21.15.79:0/1404163720 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 9) ==== 90895+0+0 (secure 0 0 0) 0x7fbb0401b070 con 0x7fbb140b3850 2024-04-26T19:52:00.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.183+0000 7fbb02ffd640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fbaf406c7c0 0x7fbaf406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.183+0000 7fbb11d9b640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fbaf406c7c0 0x7fbaf406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.183+0000 7fbb02ffd640 1 -- 172.21.15.79:0/1404163720 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fbb0408acd0 con 0x7fbb140b3850 2024-04-26T19:52:00.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.183+0000 7fbb11d9b640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fbaf406c7c0 0x7fbaf406ec80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb141f6220 tx=0x7fbb08008040 comp rx=0 tx=0).ready entity=mgr.5580 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:00.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.187+0000 7fbb02ffd640 1 -- 172.21.15.79:0/1404163720 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fbb04058b60 con 0x7fbb140b3850 2024-04-26T19:52:00.384 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.383+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 --> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb1406d900 con 0x7fbaf406c7c0 2024-04-26T19:52:00.389 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.389+0000 7fbb02ffd640 1 -- 172.21.15.79:0/1404163720 <== mgr.5580 v2:172.21.15.79:6832/3922824612 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+72 (secure 0 0 0) 0x7fbb1406d900 con 0x7fbaf406c7c0 2024-04-26T19:52:00.390 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:00.390 INFO:teuthology.orchestra.run.smithi079.stdout:{"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fbaf406c7c0 msgr2=0x7fbaf406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fbaf406c7c0 0x7fbaf406ec80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbb141f6220 tx=0x7fbb08008040 comp rx=0 tx=0).stop 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb140b3850 msgr2=0x7fbb14155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb140b3850 0x7fbb14155bb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400c500 tx=0x7fbb0400c9c0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 shutdown_connections 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fbaf406c7c0 0x7fbaf406ec80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbb141605f0 0x7fbb141e5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbb141564e0 0x7fbb1415b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 --2- 172.21.15.79:0/1404163720 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbb140b3850 0x7fbb14155bb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 >> 172.21.15.79:0/1404163720 conn(0x7fbb14068180 msgr2=0x7fbb1406f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 shutdown_connections 2024-04-26T19:52:00.392 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.391+0000 7fbb18ea0640 1 -- 172.21.15.79:0/1404163720 wait complete. 2024-04-26T19:52:00.402 DEBUG:tasks.cephfs.test_snap_schedules:res={"path": "snap_test_dir1", "schedule": ["1m"], "retention": [{"m": 1}]} 2024-04-26T19:52:00.402 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1m'], 'retention': [{'m': 1}]} 2024-04-26T19:52:00.403 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2024-04-26T19:52:00.485 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:52:00.487 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:52:00.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.501+0000 7f5d0e714640 1 Processor -- start 2024-04-26T19:52:00.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.501+0000 7f5d0e714640 1 -- start start 2024-04-26T19:52:00.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.502+0000 7f5d0e714640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5d0813eff0 0x7f5d0806f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.502+0000 7f5d0e714640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d08142410 con 0x7f5d08141f10 2024-04-26T19:52:00.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.502+0000 7f5d0e714640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d08072a90 con 0x7f5d08073570 2024-04-26T19:52:00.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.502+0000 7f5d0e714640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f5d081426f0 con 0x7f5d0813eff0 2024-04-26T19:52:00.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.502+0000 7f5d07fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5d0813eff0 0x7f5d0806f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.502+0000 7f5d0ccb1640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5d08141f10 0x7f5d0813f770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59720/0 (socket says 172.21.15.79:59720) 2024-04-26T19:52:00.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.502+0000 7f5d0ccb1640 1 -- 172.21.15.79:0/869660066 learned_addr learned my addr 172.21.15.79:0/869660066 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:00.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.503+0000 7f5d07fff640 1 -- 172.21.15.79:0/869660066 >> v1:172.21.15.83:6789/0 conn(0x7f5d08073570 legacy=0x7f5d08141800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.503+0000 7f5d07fff640 1 -- 172.21.15.79:0/869660066 >> v1:172.21.15.79:6789/0 conn(0x7f5d08141f10 legacy=0x7f5d0813f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.503+0000 7f5d07fff640 1 -- 172.21.15.79:0/869660066 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d08072d70 con 0x7f5d0813eff0 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.503+0000 7f5d07fff640 1 --2- 172.21.15.79:0/869660066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d0813eff0 0x7f5d0806f0a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5d08142870 tx=0x7f5cfc02c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c65615d8e2137603 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.503+0000 7f5d06ffd640 1 -- 172.21.15.79:0/869660066 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc039070 con 0x7f5d0813eff0 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.503+0000 7f5d06ffd640 1 -- 172.21.15.79:0/869660066 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cfc034640 con 0x7f5d0813eff0 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.503+0000 7f5d06ffd640 1 -- 172.21.15.79:0/869660066 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc034a10 con 0x7f5d0813eff0 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.504+0000 7f5d0e714640 1 -- 172.21.15.79:0/869660066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d0813eff0 msgr2=0x7f5d0806f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.504+0000 7f5d0e714640 1 --2- 172.21.15.79:0/869660066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d0813eff0 0x7f5d0806f0a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5d08142870 tx=0x7f5cfc02c390 comp rx=0 tx=0).stop 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.504+0000 7f5d0e714640 1 -- 172.21.15.79:0/869660066 shutdown_connections 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.504+0000 7f5d0e714640 1 --2- 172.21.15.79:0/869660066 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d0813eff0 0x7f5d0806f0a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.504+0000 7f5d0e714640 1 -- 172.21.15.79:0/869660066 >> 172.21.15.79:0/869660066 conn(0x7f5d08068860 msgr2=0x7f5d080b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.504+0000 7f5d0e714640 1 -- 172.21.15.79:0/869660066 shutdown_connections 2024-04-26T19:52:00.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.504+0000 7f5d0e714640 1 -- 172.21.15.79:0/869660066 wait complete. 2024-04-26T19:52:00.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.505+0000 7f5d0e714640 1 Processor -- start 2024-04-26T19:52:00.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.505+0000 7f5d0e714640 1 -- start start 2024-04-26T19:52:00.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d0e714640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5d08073570 0x7f5d08140fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d0e714640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0813eff0 0x7f5d081f5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d0e714640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d08141f10 0x7f5d08159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d0e714640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0813f3d0 con 0x7f5d08141f10 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d0e714640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08074770 con 0x7f5d0813eff0 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d0e714640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0806f5e0 con 0x7f5d08073570 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d0ccb1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d08141f10 0x7f5d08159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d0ccb1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d08141f10 0x7f5d08159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54544/0 (socket says 172.21.15.79:54544) 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d0ccb1640 1 -- 172.21.15.79:0/3895601144 learned_addr learned my addr 172.21.15.79:0/3895601144 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d077fe640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0813eff0 0x7f5d081f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.506+0000 7f5d07fff640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5d08073570 0x7f5d08140fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d0ccb1640 1 -- 172.21.15.79:0/3895601144 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5d08073570 msgr2=0x7f5d08140fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d0ccb1640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5d08073570 0x7f5d08140fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.510 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d0ccb1640 1 -- 172.21.15.79:0/3895601144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0813eff0 msgr2=0x7f5d081f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d0ccb1640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0813eff0 0x7f5d081f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d0ccb1640 1 -- 172.21.15.79:0/3895601144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0813e1c0 con 0x7f5d08141f10 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d0ccb1640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d08141f10 0x7f5d08159200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000caa0 tx=0x7f5d0000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d04ff9640 1 -- 172.21.15.79:0/3895601144 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00017070 con 0x7f5d08141f10 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d04ff9640 1 -- 172.21.15.79:0/3895601144 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d0000aba0 con 0x7f5d08141f10 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d04ff9640 1 -- 172.21.15.79:0/3895601144 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d00012400 con 0x7f5d08141f10 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d0813e430 con 0x7f5d08141f10 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.507+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d08156470 con 0x7f5d08141f10 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.508+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d080506c0 con 0x7f5d08141f10 2024-04-26T19:52:00.511 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.509+0000 7f5d04ff9640 1 -- 172.21.15.79:0/3895601144 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 9) ==== 90895+0+0 (secure 0 0 0) 0x7f5d0001a070 con 0x7f5d08141f10 2024-04-26T19:52:00.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.509+0000 7f5d04ff9640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f5cf406c790 0x7f5cf406ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:00.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.509+0000 7f5d07fff640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f5cf406c790 0x7f5cf406ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:00.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.509+0000 7f5d04ff9640 1 -- 172.21.15.79:0/3895601144 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f5d0008b590 con 0x7f5d08141f10 2024-04-26T19:52:00.512 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.510+0000 7f5d07fff640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f5cf406c790 0x7f5cf406ec50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc036000 tx=0x7f5cfc030000 comp rx=0 tx=0).ready entity=mgr.5580 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:00.514 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.514+0000 7f5d04ff9640 1 -- 172.21.15.79:0/3895601144 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5d00059420 con 0x7f5d08141f10 2024-04-26T19:52:00.591 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.590+0000 7f5d04ff9640 1 -- 172.21.15.79:0/3895601144 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 10) ==== 91160+0+0 (secure 0 0 0) 0x7f5d00055a90 con 0x7f5d08141f10 2024-04-26T19:52:00.723 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.722+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 --> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) -- 0x7f5d08156a50 con 0x7f5cf406c790 2024-04-26T19:52:00.783 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.783+0000 7f5d04ff9640 1 -- 172.21.15.79:0/3895601144 <== mgr.5580 v2:172.21.15.79:6832/3922824612 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f5d08156a50 con 0x7f5cf406c790 2024-04-26T19:52:00.783 INFO:teuthology.orchestra.run.smithi079.stdout:Schedule removed for path snap_test_dir1 2024-04-26T19:52:00.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f5cf406c790 msgr2=0x7f5cf406ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f5cf406c790 0x7f5cf406ec50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc036000 tx=0x7f5cfc030000 comp rx=0 tx=0).stop 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d08141f10 msgr2=0x7f5d08159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d08141f10 0x7f5d08159200 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000caa0 tx=0x7f5d0000cf60 comp rx=0 tx=0).stop 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 shutdown_connections 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f5cf406c790 0x7f5cf406ec50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5d08141f10 0x7f5d08159200 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5d0813eff0 0x7f5d081f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 --2- 172.21.15.79:0/3895601144 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5d08073570 0x7f5d08140fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 >> 172.21.15.79:0/3895601144 conn(0x7f5d08068860 msgr2=0x7f5d08160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 shutdown_connections 2024-04-26T19:52:00.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:00.785+0000 7f5d0e714640 1 -- 172.21.15.79:0/3895601144 wait complete. 2024-04-26T19:52:00.796 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2024-04-26T19:52:00.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:00.796 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.customsnapkernel) 2024-04-26T19:52:00.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:00.812 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2024-04-26T19:52:00.869 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2024-04-26T19:52:01.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.007+0000 7f913d69b640 1 Processor -- start 2024-04-26T19:52:01.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.007+0000 7f913d69b640 1 -- start start 2024-04-26T19:52:01.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.008+0000 7f913d69b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9138073d80 0x7f9138074200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.008+0000 7f913d69b640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f91380af550 0x7f91380af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.008+0000 7f913d69b640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91380d30f0 con 0x7f913815d0f0 2024-04-26T19:52:01.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.008+0000 7f913d69b640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f91380af0c0 con 0x7f91380af550 2024-04-26T19:52:01.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.008+0000 7f913d69b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f913815d4d0 con 0x7f9138073d80 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.008+0000 7f9136ffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f913815d0f0 0x7f91380d3d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59730/0 (socket says 172.21.15.79:59730) 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.008+0000 7f9136ffd640 1 -- 172.21.15.79:0/3978438849 learned_addr learned my addr 172.21.15.79:0/3978438849 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f91377fe640 1 --2- 172.21.15.79:0/3978438849 >> v2:172.21.15.83:3300/0 conn(0x7f91380af550 0x7f91380af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f91367fc640 1 --2- 172.21.15.79:0/3978438849 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9138073d80 0x7f9138074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f9135ffb640 1 -- 172.21.15.79:0/3978438849 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787469925 0 0) 0x7f9128001490 con 0x7f913815d0f0 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f9135ffb640 1 -- 172.21.15.79:0/3978438849 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9120003520 con 0x7f913815d0f0 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f9135ffb640 1 -- 172.21.15.79:0/3978438849 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3979712668 0 0) 0x7f9120003520 con 0x7f913815d0f0 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f91377fe640 1 -- 172.21.15.79:0/3978438849 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9138073d80 msgr2=0x7f9138074200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f91377fe640 1 --2- 172.21.15.79:0/3978438849 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9138073d80 0x7f9138074200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f91377fe640 1 -- 172.21.15.79:0/3978438849 >> v1:172.21.15.79:6789/0 conn(0x7f913815d0f0 legacy=0x7f91380d3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f91377fe640 1 -- 172.21.15.79:0/3978438849 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91380aff00 con 0x7f91380af550 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.009+0000 7f91367fc640 1 --2- 172.21.15.79:0/3978438849 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9138073d80 0x7f9138074200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:01.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f91377fe640 1 --2- 172.21.15.79:0/3978438849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91380af550 0x7f91380af9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f913002a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70c42ab57a90f4e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:01.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f9135ffb640 1 -- 172.21.15.79:0/3978438849 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130037070 con 0x7f91380af550 2024-04-26T19:52:01.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f9135ffb640 1 -- 172.21.15.79:0/3978438849 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f913002aba0 con 0x7f91380af550 2024-04-26T19:52:01.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f9135ffb640 1 -- 172.21.15.79:0/3978438849 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9128001bc0 con 0x7f91380af550 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f913d69b640 1 -- 172.21.15.79:0/3978438849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91380af550 msgr2=0x7f91380af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f913d69b640 1 --2- 172.21.15.79:0/3978438849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91380af550 0x7f91380af9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f913002a110 comp rx=0 tx=0).stop 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f913d69b640 1 -- 172.21.15.79:0/3978438849 shutdown_connections 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f913d69b640 1 --2- 172.21.15.79:0/3978438849 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f91380af550 0x7f91380af9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f913d69b640 1 --2- 172.21.15.79:0/3978438849 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9138073d80 0x7f9138074200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f913d69b640 1 -- 172.21.15.79:0/3978438849 >> 172.21.15.79:0/3978438849 conn(0x7f91380687b0 msgr2=0x7f9138071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f913d69b640 1 -- 172.21.15.79:0/3978438849 shutdown_connections 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.010+0000 7f913d69b640 1 -- 172.21.15.79:0/3978438849 wait complete. 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f913d69b640 1 Processor -- start 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f913d69b640 1 -- start start 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f913d69b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9138073d80 0x7f91381565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f913d69b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f91380af550 0x7f9138157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f913d69b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f913815d0f0 0x7f9138161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f913d69b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f91380aed50 con 0x7f913815d0f0 2024-04-26T19:52:01.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f913d69b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f91380aea70 con 0x7f9138073d80 2024-04-26T19:52:01.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f913d69b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f91380d4430 con 0x7f91380af550 2024-04-26T19:52:01.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f91377fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f913815d0f0 0x7f9138161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f91377fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f913815d0f0 0x7f9138161230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54570/0 (socket says 172.21.15.79:54570) 2024-04-26T19:52:01.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f91377fe640 1 -- 172.21.15.79:0/520114087 learned_addr learned my addr 172.21.15.79:0/520114087 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:01.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.012+0000 7f9136ffd640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9138073d80 0x7f91381565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f91367fc640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f91380af550 0x7f9138157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f91377fe640 1 -- 172.21.15.79:0/520114087 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f91380af550 msgr2=0x7f9138157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f91377fe640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f91380af550 0x7f9138157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f91377fe640 1 -- 172.21.15.79:0/520114087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9138073d80 msgr2=0x7f91381565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f91377fe640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9138073d80 0x7f91381565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f91377fe640 1 -- 172.21.15.79:0/520114087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9138156e90 con 0x7f913815d0f0 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f91377fe640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f913815d0f0 0x7f9138161230 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f91300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f911f7fe640 1 -- 172.21.15.79:0/520114087 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130037050 con 0x7f913815d0f0 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f911f7fe640 1 -- 172.21.15.79:0/520114087 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9130038040 con 0x7f913815d0f0 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f911f7fe640 1 -- 172.21.15.79:0/520114087 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9130006e10 con 0x7f913815d0f0 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9138157170 con 0x7f913815d0f0 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.013+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9138157730 con 0x7f913815d0f0 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.014+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91380506c0 con 0x7f913815d0f0 2024-04-26T19:52:01.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.014+0000 7f911f7fe640 1 -- 172.21.15.79:0/520114087 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 10) ==== 91160+0+0 (secure 0 0 0) 0x7f9130003610 con 0x7f913815d0f0 2024-04-26T19:52:01.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.015+0000 7f911f7fe640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f912406ca60 0x7f912406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.015+0000 7f911f7fe640 1 -- 172.21.15.79:0/520114087 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f91300ac6f0 con 0x7f913815d0f0 2024-04-26T19:52:01.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.015+0000 7f9136ffd640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f912406ca60 0x7f912406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.015+0000 7f9136ffd640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f912406ca60 0x7f912406ef20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9128005c00 tx=0x7f9128009040 comp rx=0 tx=0).ready entity=mgr.5580 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:01.023 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.019+0000 7f911f7fe640 1 -- 172.21.15.79:0/520114087 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f913007a780 con 0x7f913815d0f0 2024-04-26T19:52:01.239 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.239+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) -- 0x7f91380d3700 con 0x7f913815d0f0 2024-04-26T19:52:01.594 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.593+0000 7f911f7fe640 1 -- 172.21.15.79:0/520114087 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v11) ==== 93+0+0 (secure 0 0 0) 0x7f913007a080 con 0x7f913815d0f0 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f912406ca60 msgr2=0x7f912406ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f912406ca60 0x7f912406ef20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9128005c00 tx=0x7f9128009040 comp rx=0 tx=0).stop 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f913815d0f0 msgr2=0x7f9138161230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f913815d0f0 0x7f9138161230 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f91300097a0 tx=0x7f91300046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 shutdown_connections 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f912406ca60 0x7f912406ef20 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9128005c00 tx=0x7f9128009040 comp rx=0 tx=0).stop 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f913815d0f0 0x7f9138161230 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f91380af550 0x7f9138157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 --2- 172.21.15.79:0/520114087 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9138073d80 0x7f91381565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.596+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 >> 172.21.15.79:0/520114087 conn(0x7f91380687b0 msgr2=0x7f91380d3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.597+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 shutdown_connections 2024-04-26T19:52:01.598 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.597+0000 7f913d69b640 1 -- 172.21.15.79:0/520114087 wait complete. 2024-04-26T19:52:01.611 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-26T19:52:01.611 DEBUG:teuthology.orchestra.run.smithi079:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T19:52:01.651 DEBUG:teuthology.orchestra.run.smithi083:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-26T19:52:01.688 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-26T19:52:01.688 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T19:52:01.688 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:52:01.688 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:52:01.707 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:52:01.707 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:52:01.711 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:01.710+0000 7fc7ce920040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:52:01.715 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:01.714+0000 7f7213dce040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T19:52:01.726 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:52:01.828 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:01.827+0000 7f7213dce040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:52:01.829 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:01.828+0000 7fc7ce920040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:52:01.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.863+0000 7f205c546640 1 Processor -- start 2024-04-26T19:52:01.864 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.863+0000 7f205c546640 1 -- start start 2024-04-26T19:52:01.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.864+0000 7f205c546640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2054159c70 0x7f2054163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.864+0000 7f205c546640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f20540b3b40 0x7f20540b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.864+0000 7f205c546640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2054160280 con 0x7f2054159890 2024-04-26T19:52:01.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.864+0000 7f205c546640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f20540b44f0 con 0x7f20540b3b40 2024-04-26T19:52:01.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.864+0000 7f205c546640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f2054160560 con 0x7f2054159c70 2024-04-26T19:52:01.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f205a2e2640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f2054159890 0x7f205415fa20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59744/0 (socket says 172.21.15.79:59744) 2024-04-26T19:52:01.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f205a2e2640 1 -- 172.21.15.79:0/1468647067 learned_addr learned my addr 172.21.15.79:0/1468647067 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:01.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f2059ae1640 1 --2- 172.21.15.79:0/1468647067 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2054159c70 0x7f2054163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.866 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f205aae3640 1 --2- 172.21.15.79:0/1468647067 >> v2:172.21.15.83:3300/0 conn(0x7f20540b3b40 0x7f20540b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f20592e0640 1 -- 172.21.15.79:0/1468647067 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3375640111 0 0) 0x7f2054160280 con 0x7f2054159890 2024-04-26T19:52:01.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f20592e0640 1 -- 172.21.15.79:0/1468647067 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2040003520 con 0x7f2054159890 2024-04-26T19:52:01.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f205aae3640 1 -- 172.21.15.79:0/1468647067 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2054159c70 msgr2=0x7f2054163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:01.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f205aae3640 1 --2- 172.21.15.79:0/1468647067 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2054159c70 0x7f2054163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f205aae3640 1 -- 172.21.15.79:0/1468647067 >> v1:172.21.15.79:6789/0 conn(0x7f2054159890 legacy=0x7f205415fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f205aae3640 1 -- 172.21.15.79:0/1468647067 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2054160840 con 0x7f20540b3b40 2024-04-26T19:52:01.867 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.865+0000 7f2059ae1640 1 --2- 172.21.15.79:0/1468647067 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2054159c70 0x7f2054163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:01.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.866+0000 7f205aae3640 1 --2- 172.21.15.79:0/1468647067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20540b3b40 0x7f20540b3fb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f204c009d90 tx=0x7f204c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4736ed87b16e4f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:01.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.866+0000 7f20592e0640 1 -- 172.21.15.79:0/1468647067 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c007740 con 0x7f20540b3b40 2024-04-26T19:52:01.870 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.866+0000 7f20592e0640 1 -- 172.21.15.79:0/1468647067 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204c0078e0 con 0x7f20540b3b40 2024-04-26T19:52:01.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.866+0000 7f20592e0640 1 -- 172.21.15.79:0/1468647067 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204c007c90 con 0x7f20540b3b40 2024-04-26T19:52:01.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.867+0000 7f205c546640 1 -- 172.21.15.79:0/1468647067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20540b3b40 msgr2=0x7f20540b3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.867+0000 7f205c546640 1 --2- 172.21.15.79:0/1468647067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20540b3b40 0x7f20540b3fb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f204c009d90 tx=0x7f204c02a660 comp rx=0 tx=0).stop 2024-04-26T19:52:01.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.867+0000 7f205c546640 1 -- 172.21.15.79:0/1468647067 shutdown_connections 2024-04-26T19:52:01.871 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.867+0000 7f205c546640 1 --2- 172.21.15.79:0/1468647067 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f20540b3b40 0x7f20540b3fb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.867+0000 7f205c546640 1 --2- 172.21.15.79:0/1468647067 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2054159c70 0x7f2054163a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.867+0000 7f205c546640 1 -- 172.21.15.79:0/1468647067 >> 172.21.15.79:0/1468647067 conn(0x7f20540b2720 msgr2=0x7f2054070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.867+0000 7f205c546640 1 -- 172.21.15.79:0/1468647067 shutdown_connections 2024-04-26T19:52:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.867+0000 7f205c546640 1 -- 172.21.15.79:0/1468647067 wait complete. 2024-04-26T19:52:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.868+0000 7f205c546640 1 Processor -- start 2024-04-26T19:52:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.868+0000 7f205c546640 1 -- start start 2024-04-26T19:52:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205c546640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f20540b3b40 0x7f205413ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205c546640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2054159890 0x7f205415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.872 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205c546640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054159c70 0x7f205415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205c546640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2054161340 con 0x7f2054159890 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205c546640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2054160b20 con 0x7f2054159c70 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205c546640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f20541614c0 con 0x7f20540b3b40 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f2059ae1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2054159890 0x7f205415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f2059ae1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2054159890 0x7f205415a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54592/0 (socket says 172.21.15.79:54592) 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f2059ae1640 1 -- 172.21.15.79:0/3688718962 learned_addr learned my addr 172.21.15.79:0/3688718962 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205a2e2640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f20540b3b40 0x7f205413ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205aae3640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054159c70 0x7f205415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f2059ae1640 1 -- 172.21.15.79:0/3688718962 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f20540b3b40 msgr2=0x7f205413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f2059ae1640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f20540b3b40 0x7f205413ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f2059ae1640 1 -- 172.21.15.79:0/3688718962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054159c70 msgr2=0x7f205415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f2059ae1640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054159c70 0x7f205415a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f2059ae1640 1 -- 172.21.15.79:0/3688718962 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205414f270 con 0x7f2054159890 2024-04-26T19:52:01.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f2059ae1640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2054159890 0x7f205415a080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f204400e930 tx=0x7f204400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:01.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f204affd640 1 -- 172.21.15.79:0/3688718962 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2044017070 con 0x7f2054159890 2024-04-26T19:52:01.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f204affd640 1 -- 172.21.15.79:0/3688718962 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f204400ca00 con 0x7f2054159890 2024-04-26T19:52:01.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f204affd640 1 -- 172.21.15.79:0/3688718962 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204400cdf0 con 0x7f2054159890 2024-04-26T19:52:01.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2054141b40 con 0x7f2054159890 2024-04-26T19:52:01.874 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.869+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205406e7c0 con 0x7f2054159890 2024-04-26T19:52:01.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.870+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20540506c0 con 0x7f2054159890 2024-04-26T19:52:01.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.871+0000 7f204affd640 1 -- 172.21.15.79:0/3688718962 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7f204401a070 con 0x7f2054159890 2024-04-26T19:52:01.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.871+0000 7f204affd640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f203806ca10 0x7f203806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:01.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.872+0000 7f205a2e2640 1 -- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f203806ca10 msgr2=0x7f203806eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:01.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.872+0000 7f204affd640 1 -- 172.21.15.79:0/3688718962 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f204408b970 con 0x7f2054159890 2024-04-26T19:52:01.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.872+0000 7f205a2e2640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f203806ca10 0x7f203806eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:01.876 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:01.874+0000 7f7213dce040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:52:01.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:01.876+0000 7f204affd640 1 -- 172.21.15.79:0/3688718962 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2044059440 con 0x7f2054159890 2024-04-26T19:52:01.877 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:01.877+0000 7fc7ce920040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T19:52:02.061 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.061+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2054070de0 con 0x7f2054159890 2024-04-26T19:52:02.062 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.062+0000 7f204affd640 1 -- 172.21.15.79:0/3688718962 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17269 (secure 0 0 0) 0x7f204405d740 con 0x7f2054159890 2024-04-26T19:52:02.063 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:02.063 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":40,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:58.575840+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:51:41.583741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T19:51:42.339687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:0/16732380":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1084811462":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/3742475428":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1948040587":"2024-04-27T19:51:58.575815+0000","172.21.15.79:6833/3161939478":"2024-04-27T19:51:58.575815+0000","172.21.15.79:6832/3161939478":"2024-04-27T19:51:58.575815+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f203806ca10 msgr2=0x7f203806eed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f203806ca10 0x7f203806eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2054159890 msgr2=0x7f205415a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2054159890 0x7f205415a080 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f204400e930 tx=0x7f204400edf0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 shutdown_connections 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f203806ca10 0x7f203806eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2054159c70 0x7f205415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2054159890 0x7f205415a080 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 --2- 172.21.15.79:0/3688718962 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f20540b3b40 0x7f205413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 >> 172.21.15.79:0/3688718962 conn(0x7f20540b2720 msgr2=0x7f205406cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:02.065 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 shutdown_connections 2024-04-26T19:52:02.066 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.064+0000 7f205c546640 1 -- 172.21.15.79:0/3688718962 wait complete. 2024-04-26T19:52:02.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:51:41.583741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:51:42.339687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T19:52:02.076 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:52:02.078 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:02.077+0000 7f7213dce040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:52:02.091 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:02.090+0000 7fc7ce920040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T19:52:02.120 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:02.119+0000 7f7213dce040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:52:02.136 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:02.136+0000 7fc7ce920040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T19:52:02.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.178+0000 7fac04d78640 1 Processor -- start 2024-04-26T19:52:02.179 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.178+0000 7fac04d78640 1 -- start start 2024-04-26T19:52:02.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fac04d78640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fac00073930 0x7fac0015d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fac04d78640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac0015fc80 0x7fac00072960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fac04d78640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac00160420 con 0x7fac0006f0e0 2024-04-26T19:52:02.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fac04d78640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fac00160700 con 0x7fac00073930 2024-04-26T19:52:02.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fac04d78640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fac001609e0 con 0x7fac0015fc80 2024-04-26T19:52:02.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fabfe59c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fac00073930 0x7fac0015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.180 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fabfe59c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fac00073930 0x7fac0015d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54606/0 (socket says 172.21.15.79:54606) 2024-04-26T19:52:02.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fabfe59c640 1 -- 172.21.15.79:0/2212952857 learned_addr learned my addr 172.21.15.79:0/2212952857 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:02.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fabfdd9b640 1 --2- 172.21.15.79:0/2212952857 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac0015fc80 0x7fac00072960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fabfd59a640 1 -- 172.21.15.79:0/2212952857 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2403739649 0 0) 0x7fac00160420 con 0x7fac0006f0e0 2024-04-26T19:52:02.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fabfe59c640 1 -- 172.21.15.79:0/2212952857 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac0015fc80 msgr2=0x7fac00072960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:02.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fabfe59c640 1 --2- 172.21.15.79:0/2212952857 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac0015fc80 0x7fac00072960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.179+0000 7fabfe59c640 1 -- 172.21.15.79:0/2212952857 >> v1:172.21.15.83:6789/0 conn(0x7fac0006f0e0 legacy=0x7fac000d0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.181 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.180+0000 7fabfe59c640 1 -- 172.21.15.79:0/2212952857 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac000d08e0 con 0x7fac00073930 2024-04-26T19:52:02.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.180+0000 7fabfe59c640 1 --2- 172.21.15.79:0/2212952857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac00073930 0x7fac0015d230 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009870 tx=0x7fabf402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480a3d481052df4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:02.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.180+0000 7fabfd59a640 1 -- 172.21.15.79:0/2212952857 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4037070 con 0x7fac00073930 2024-04-26T19:52:02.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.180+0000 7fabfd59a640 1 -- 172.21.15.79:0/2212952857 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf4032440 con 0x7fac00073930 2024-04-26T19:52:02.182 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.180+0000 7fabfd59a640 1 -- 172.21.15.79:0/2212952857 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4032810 con 0x7fac00073930 2024-04-26T19:52:02.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.181+0000 7fac04d78640 1 -- 172.21.15.79:0/2212952857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac00073930 msgr2=0x7fac0015d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.181+0000 7fac04d78640 1 --2- 172.21.15.79:0/2212952857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac00073930 0x7fac0015d230 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009870 tx=0x7fabf402a270 comp rx=0 tx=0).stop 2024-04-26T19:52:02.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.181+0000 7fac04d78640 1 -- 172.21.15.79:0/2212952857 shutdown_connections 2024-04-26T19:52:02.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.181+0000 7fac04d78640 1 --2- 172.21.15.79:0/2212952857 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac0015fc80 0x7fac00072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.181+0000 7fac04d78640 1 --2- 172.21.15.79:0/2212952857 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac00073930 0x7fac0015d230 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.181+0000 7fac04d78640 1 -- 172.21.15.79:0/2212952857 >> 172.21.15.79:0/2212952857 conn(0x7fac000689e0 msgr2=0x7fac000b3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:02.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.181+0000 7fac04d78640 1 -- 172.21.15.79:0/2212952857 shutdown_connections 2024-04-26T19:52:02.183 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.181+0000 7fac04d78640 1 -- 172.21.15.79:0/2212952857 wait complete. 2024-04-26T19:52:02.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.182+0000 7fac04d78640 1 Processor -- start 2024-04-26T19:52:02.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.182+0000 7fac04d78640 1 -- start start 2024-04-26T19:52:02.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fac04d78640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac0006f0e0 0x7fac001f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fac04d78640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac00073930 0x7fac00158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fac04d78640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0015fc80 0x7fac000b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fac04d78640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac00072ea0 con 0x7fac0006f0e0 2024-04-26T19:52:02.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fac04d78640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac00073d10 con 0x7fac0015fc80 2024-04-26T19:52:02.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fac04d78640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fac00073020 con 0x7fac00073930 2024-04-26T19:52:02.185 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fabfe59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac0006f0e0 0x7fac001f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fabfe59c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac0006f0e0 0x7fac001f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54612/0 (socket says 172.21.15.79:54612) 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fabfe59c640 1 -- 172.21.15.79:0/438614404 learned_addr learned my addr 172.21.15.79:0/438614404 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fabfed9d640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0015fc80 0x7fac000b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.183+0000 7fabfdd9b640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac00073930 0x7fac00158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fabfe59c640 1 -- 172.21.15.79:0/438614404 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac00073930 msgr2=0x7fac00158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fabfe59c640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac00073930 0x7fac00158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fabfe59c640 1 -- 172.21.15.79:0/438614404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0015fc80 msgr2=0x7fac000b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fabfe59c640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0015fc80 0x7fac000b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fabfe59c640 1 -- 172.21.15.79:0/438614404 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac000af520 con 0x7fac0006f0e0 2024-04-26T19:52:02.186 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fabfe59c640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac0006f0e0 0x7fac001f5270 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000c00 tx=0x7fabf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:02.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fabeeffd640 1 -- 172.21.15.79:0/438614404 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4037040 con 0x7fac0006f0e0 2024-04-26T19:52:02.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac001f9550 con 0x7fac0006f0e0 2024-04-26T19:52:02.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fabeeffd640 1 -- 172.21.15.79:0/438614404 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabf4042070 con 0x7fac0006f0e0 2024-04-26T19:52:02.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fabeeffd640 1 -- 172.21.15.79:0/438614404 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4030880 con 0x7fac0006f0e0 2024-04-26T19:52:02.187 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.184+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac000afac0 con 0x7fac0006f0e0 2024-04-26T19:52:02.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.185+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac000506c0 con 0x7fac0006f0e0 2024-04-26T19:52:02.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.186+0000 7fabeeffd640 1 -- 172.21.15.79:0/438614404 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7fabf4030a20 con 0x7fac0006f0e0 2024-04-26T19:52:02.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.186+0000 7fabeeffd640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fabe006ca10 0x7fabe006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.186+0000 7fabfdd9b640 1 -- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fabe006ca10 msgr2=0x7fabe006eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:02.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.186+0000 7fabeeffd640 1 -- 172.21.15.79:0/438614404 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fabf40ac6c0 con 0x7fac0006f0e0 2024-04-26T19:52:02.188 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.186+0000 7fabfdd9b640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fabe006ca10 0x7fabe006eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:02.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.191+0000 7fabeeffd640 1 -- 172.21.15.79:0/438614404 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fabf407a110 con 0x7fac0006f0e0 2024-04-26T19:52:02.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.372+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac000d0310 con 0x7fac0006f0e0 2024-04-26T19:52:02.373 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.372+0000 7fabeeffd640 1 -- 172.21.15.79:0/438614404 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7fabf407e410 con 0x7fac0006f0e0 2024-04-26T19:52:02.373 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:52:02.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fabe006ca10 msgr2=0x7fabe006eed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:02.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fabe006ca10 0x7fabe006eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac0006f0e0 msgr2=0x7fac001f5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac0006f0e0 0x7fac001f5270 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000c00 tx=0x7fabf40046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 shutdown_connections 2024-04-26T19:52:02.376 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fabe006ca10 0x7fabe006eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac0015fc80 0x7fac000b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac00073930 0x7fac00158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 --2- 172.21.15.79:0/438614404 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac0006f0e0 0x7fac001f5270 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 >> 172.21.15.79:0/438614404 conn(0x7fac000689e0 msgr2=0x7fac0015ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:02.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 shutdown_connections 2024-04-26T19:52:02.377 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.374+0000 7fac04d78640 1 -- 172.21.15.79:0/438614404 wait complete. 2024-04-26T19:52:02.386 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:52:02.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.529+0000 7f9f24a1d640 1 Processor -- start 2024-04-26T19:52:02.529 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.529+0000 7f9f24a1d640 1 -- start start 2024-04-26T19:52:02.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.529+0000 7f9f24a1d640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f200aeeb0 0x7f9f2006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.529+0000 7f9f24a1d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f200b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.529+0000 7f9f24a1d640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f2006fa10 con 0x7f9f200b1990 2024-04-26T19:52:02.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.529+0000 7f9f24a1d640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f9f20072ea0 con 0x7f9f200aeeb0 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.529+0000 7f9f24a1d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9f20073180 con 0x7f9f20070160 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.530+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f200b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.530+0000 7f9f1e59c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f9f200aeeb0 0x7f9f2006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.530+0000 7f9f1ed9d640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f9f200b1990 0x7f9f200af290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:58590/0 (socket says 172.21.15.79:58590) 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.530+0000 7f9f1ed9d640 1 -- 172.21.15.79:0/915477387 learned_addr learned my addr 172.21.15.79:0/915477387 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.530+0000 7f9f1e59c640 1 -- 172.21.15.79:0/915477387 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 msgr2=0x7f9f200b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.530+0000 7f9f1e59c640 1 --2- 172.21.15.79:0/915477387 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f200b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.530+0000 7f9f1e59c640 1 -- 172.21.15.79:0/915477387 >> v1:172.21.15.83:6789/0 conn(0x7f9f200b1990 legacy=0x7f9f200af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.530+0000 7f9f1e59c640 1 -- 172.21.15.79:0/915477387 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f20073460 con 0x7f9f200aeeb0 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.530+0000 7f9f17fff640 1 --2- 172.21.15.79:0/915477387 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f200b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.531+0000 7f9f1e59c640 1 --2- 172.21.15.79:0/915477387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200aeeb0 0x7f9f2006f0e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097a0 tx=0x7f9f1002a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa6daf2be45ae2ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.531+0000 7f9f1dd9b640 1 -- 172.21.15.79:0/915477387 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10037070 con 0x7f9f200aeeb0 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.531+0000 7f9f1dd9b640 1 -- 172.21.15.79:0/915477387 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f1002ae50 con 0x7f9f200aeeb0 2024-04-26T19:52:02.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.531+0000 7f9f1dd9b640 1 -- 172.21.15.79:0/915477387 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f10032660 con 0x7f9f200aeeb0 2024-04-26T19:52:02.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.531+0000 7f9f24a1d640 1 -- 172.21.15.79:0/915477387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200aeeb0 msgr2=0x7f9f2006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.531+0000 7f9f24a1d640 1 --2- 172.21.15.79:0/915477387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200aeeb0 0x7f9f2006f0e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9f100097a0 tx=0x7f9f1002a100 comp rx=0 tx=0).stop 2024-04-26T19:52:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.532+0000 7f9f24a1d640 1 -- 172.21.15.79:0/915477387 shutdown_connections 2024-04-26T19:52:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.532+0000 7f9f24a1d640 1 --2- 172.21.15.79:0/915477387 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f200b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.532+0000 7f9f24a1d640 1 --2- 172.21.15.79:0/915477387 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200aeeb0 0x7f9f2006f0e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.532+0000 7f9f24a1d640 1 -- 172.21.15.79:0/915477387 >> 172.21.15.79:0/915477387 conn(0x7f9f200687e0 msgr2=0x7f9f20071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.532+0000 7f9f24a1d640 1 -- 172.21.15.79:0/915477387 shutdown_connections 2024-04-26T19:52:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.532+0000 7f9f24a1d640 1 -- 172.21.15.79:0/915477387 wait complete. 2024-04-26T19:52:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.533+0000 7f9f24a1d640 1 Processor -- start 2024-04-26T19:52:02.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.533+0000 7f9f24a1d640 1 -- start start 2024-04-26T19:52:02.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f24a1d640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f20147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f24a1d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f200aeeb0 0x7f9f20140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f24a1d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200b1990 0x7f9f20140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f24a1d640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f9f2006f620 con 0x7f9f200aeeb0 2024-04-26T19:52:02.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f24a1d640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f9f20149380 con 0x7f9f200b1990 2024-04-26T19:52:02.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f24a1d640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f9f2006f7a0 con 0x7f9f20070160 2024-04-26T19:52:02.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f200aeeb0 0x7f9f20140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f17fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f200aeeb0 0x7f9f20140230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54622/0 (socket says 172.21.15.79:54622) 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f17fff640 1 -- 172.21.15.79:0/2600166344 learned_addr learned my addr 172.21.15.79:0/2600166344 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f1e59c640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f20147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f1ed9d640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200b1990 0x7f9f20140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f17fff640 1 -- 172.21.15.79:0/2600166344 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 msgr2=0x7f9f20147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f17fff640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f20147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f17fff640 1 -- 172.21.15.79:0/2600166344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200b1990 msgr2=0x7f9f20140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f17fff640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200b1990 0x7f9f20140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f17fff640 1 -- 172.21.15.79:0/2600166344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f20140cb0 con 0x7f9f200aeeb0 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f1e59c640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f20147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f17fff640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f200aeeb0 0x7f9f20140230 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e900 tx=0x7f9f0800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f16ffd640 1 -- 172.21.15.79:0/2600166344 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08017070 con 0x7f9f200aeeb0 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f16ffd640 1 -- 172.21.15.79:0/2600166344 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f0800ca00 con 0x7f9f200aeeb0 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f201431d0 con 0x7f9f200aeeb0 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f16ffd640 1 -- 172.21.15.79:0/2600166344 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0800cdf0 con 0x7f9f200aeeb0 2024-04-26T19:52:02.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.534+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f20143790 con 0x7f9f200aeeb0 2024-04-26T19:52:02.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.536+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f200506c0 con 0x7f9f200aeeb0 2024-04-26T19:52:02.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.536+0000 7f9f16ffd640 1 -- 172.21.15.79:0/2600166344 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7f9f0801a070 con 0x7f9f200aeeb0 2024-04-26T19:52:02.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.536+0000 7f9f16ffd640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f9f0006ca10 0x7f9f0006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.536+0000 7f9f1e59c640 1 -- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f9f0006ca10 msgr2=0x7f9f0006eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:02.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.536+0000 7f9f16ffd640 1 -- 172.21.15.79:0/2600166344 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f9f0808bea0 con 0x7f9f200aeeb0 2024-04-26T19:52:02.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.536+0000 7f9f1e59c640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f9f0006ca10 0x7f9f0006eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:02.545 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.545+0000 7f9f16ffd640 1 -- 172.21.15.79:0/2600166344 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f9f08059850 con 0x7f9f200aeeb0 2024-04-26T19:52:02.605 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:02.604+0000 7f7213dce040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:52:02.647 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:02.647+0000 7fc7ce920040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T19:52:02.690 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:02.688+0000 7f7213dce040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:52:02.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.726+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f2006a800 con 0x7f9f200aeeb0 2024-04-26T19:52:02.727 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.727+0000 7f9f16ffd640 1 -- 172.21.15.79:0/2600166344 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7f9f0805db50 con 0x7f9f200aeeb0 2024-04-26T19:52:02.727 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.728+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f9f0006ca10 msgr2=0x7f9f0006eed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f9f0006ca10 0x7f9f0006eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f200aeeb0 msgr2=0x7f9f20140230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f200aeeb0 0x7f9f20140230 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800e900 tx=0x7f9f0800edc0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 shutdown_connections 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f9f0006ca10 0x7f9f0006eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f9f200b1990 0x7f9f20140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f9f200aeeb0 0x7f9f20140230 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 --2- 172.21.15.79:0/2600166344 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f9f20070160 0x7f9f20147b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 >> 172.21.15.79:0/2600166344 conn(0x7f9f200687e0 msgr2=0x7f9f200b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 shutdown_connections 2024-04-26T19:52:02.730 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.729+0000 7f9f24a1d640 1 -- 172.21.15.79:0/2600166344 wait complete. 2024-04-26T19:52:02.731 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:02.730+0000 7f7213dce040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:52:02.734 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:02.734+0000 7fc7ce920040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T19:52:02.740 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:52:02.779 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:02.779+0000 7fc7ce920040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T19:52:02.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.841+0000 7f03321a5640 1 Processor -- start 2024-04-26T19:52:02.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.841+0000 7f03321a5640 1 -- start start 2024-04-26T19:52:02.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.842+0000 7f03321a5640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f032c146bc0 0x7f032c0b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.842+0000 7f03321a5640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f032c0d2d40 0x7f032c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.842+0000 7f03321a5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f032c0b7c60 con 0x7f032c0b91d0 2024-04-26T19:52:02.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.842+0000 7f03321a5640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f032c0b7ed0 con 0x7f032c0d2d40 2024-04-26T19:52:02.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.842+0000 7f03321a5640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f032c0b81b0 con 0x7f032c146bc0 2024-04-26T19:52:02.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.842+0000 7f0322ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f032c0d2d40 0x7f032c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.844 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.842+0000 7f0322ffd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f032c0d2d40 0x7f032c0b8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54632/0 (socket says 172.21.15.79:54632) 2024-04-26T19:52:02.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.842+0000 7f0322ffd640 1 -- 172.21.15.79:0/2378749382 learned_addr learned my addr 172.21.15.79:0/2378749382 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:02.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.842+0000 7f032b7fe640 1 --2- 172.21.15.79:0/2378749382 >> v2:172.21.15.83:3300/0 conn(0x7f032c146bc0 0x7f032c0b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.843+0000 7f0322ffd640 1 -- 172.21.15.79:0/2378749382 >> v2:172.21.15.83:3300/0 conn(0x7f032c146bc0 msgr2=0x7f032c0b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:02.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.843+0000 7f0322ffd640 1 --2- 172.21.15.79:0/2378749382 >> v2:172.21.15.83:3300/0 conn(0x7f032c146bc0 0x7f032c0b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.843+0000 7f0322ffd640 1 -- 172.21.15.79:0/2378749382 >> v1:172.21.15.83:6789/0 conn(0x7f032c0b91d0 legacy=0x7f032c0b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.843+0000 7f0322ffd640 1 -- 172.21.15.79:0/2378749382 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c0b8490 con 0x7f032c0d2d40 2024-04-26T19:52:02.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.843+0000 7f0322ffd640 1 --2- 172.21.15.79:0/2378749382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 0x7f032c0b8c90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0314009870 tx=0x7f031402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef242daaa3f2ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:02.845 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.843+0000 7f032affd640 1 -- 172.21.15.79:0/2378749382 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314037070 con 0x7f032c0d2d40 2024-04-26T19:52:02.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f032affd640 1 -- 172.21.15.79:0/2378749382 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0314032440 con 0x7f032c0d2d40 2024-04-26T19:52:02.846 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f032affd640 1 -- 172.21.15.79:0/2378749382 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0314032810 con 0x7f032c0d2d40 2024-04-26T19:52:02.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f03321a5640 1 -- 172.21.15.79:0/2378749382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 msgr2=0x7f032c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f03321a5640 1 --2- 172.21.15.79:0/2378749382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 0x7f032c0b8c90 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0314009870 tx=0x7f031402a210 comp rx=0 tx=0).stop 2024-04-26T19:52:02.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f03321a5640 1 -- 172.21.15.79:0/2378749382 shutdown_connections 2024-04-26T19:52:02.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f03321a5640 1 --2- 172.21.15.79:0/2378749382 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 0x7f032c0b8c90 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f03321a5640 1 --2- 172.21.15.79:0/2378749382 >> v2:172.21.15.83:3300/0 conn(0x7f032c146bc0 0x7f032c0b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f03321a5640 1 -- 172.21.15.79:0/2378749382 >> 172.21.15.79:0/2378749382 conn(0x7f032c0b4500 msgr2=0x7f032c0b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:02.848 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f03321a5640 1 -- 172.21.15.79:0/2378749382 shutdown_connections 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.844+0000 7f03321a5640 1 -- 172.21.15.79:0/2378749382 wait complete. 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.845+0000 7f03321a5640 1 Processor -- start 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f03321a5640 1 -- start start 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f03321a5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 0x7f032c1f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f03321a5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032c146bc0 0x7f032c142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f03321a5640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f032c1425a0 0x7f032c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f03321a5640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f032c0b73c0 con 0x7f032c0d2d40 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f03321a5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f032c146fa0 con 0x7f032c146bc0 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f03321a5640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f032c0b7540 con 0x7f032c1425a0 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 0x7f032c1f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f032b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 0x7f032c1f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54644/0 (socket says 172.21.15.79:54644) 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f032b7fe640 1 -- 172.21.15.79:0/2019773248 learned_addr learned my addr 172.21.15.79:0/2019773248 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f0322ffd640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032c146bc0 0x7f032c142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.846+0000 7f032bfff640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f032c1425a0 0x7f032c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:02.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f032b7fe640 1 -- 172.21.15.79:0/2019773248 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f032c1425a0 msgr2=0x7f032c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f032b7fe640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f032c1425a0 0x7f032c0aec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f032b7fe640 1 -- 172.21.15.79:0/2019773248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032c146bc0 msgr2=0x7f032c142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f032b7fe640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032c146bc0 0x7f032c142060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f032b7fe640 1 -- 172.21.15.79:0/2019773248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c0af8b0 con 0x7f032c0d2d40 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f0322ffd640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032c146bc0 0x7f032c142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f032b7fe640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 0x7f032c1f2da0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f031c00e8a0 tx=0x7f031c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f0328ff9640 1 -- 172.21.15.79:0/2019773248 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c017070 con 0x7f032c0d2d40 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f0328ff9640 1 -- 172.21.15.79:0/2019773248 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f031c00c940 con 0x7f032c0d2d40 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c0afb90 con 0x7f032c0d2d40 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f0328ff9640 1 -- 172.21.15.79:0/2019773248 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f031c00cd30 con 0x7f032c0d2d40 2024-04-26T19:52:02.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.847+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c0b0150 con 0x7f032c0d2d40 2024-04-26T19:52:02.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.848+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032c069390 con 0x7f032c0d2d40 2024-04-26T19:52:02.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.849+0000 7f0328ff9640 1 -- 172.21.15.79:0/2019773248 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7f031c01a070 con 0x7f032c0d2d40 2024-04-26T19:52:02.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.849+0000 7f0328ff9640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f030c06ca10 0x7f030c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:02.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.849+0000 7f0322ffd640 1 -- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f030c06ca10 msgr2=0x7f030c06eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:02.851 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.849+0000 7f0328ff9640 1 -- 172.21.15.79:0/2019773248 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f031c08bd70 con 0x7f032c0d2d40 2024-04-26T19:52:02.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.849+0000 7f0322ffd640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f030c06ca10 0x7f030c06eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:02.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:02.853+0000 7f0328ff9640 1 -- 172.21.15.79:0/2019773248 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f032c069390 con 0x7f032c0d2d40 2024-04-26T19:52:02.893 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:02.892+0000 7f7213dce040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:52:02.937 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:02.935+0000 7f7213dce040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:52:02.950 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:02.950+0000 7fc7ce920040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T19:52:02.984 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:02.983+0000 7f7213dce040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:52:02.996 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:02.995+0000 7fc7ce920040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T19:52:03.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.035+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f032c0b8c90 con 0x7f032c0d2d40 2024-04-26T19:52:03.036 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.036+0000 7f0328ff9640 1 -- 172.21.15.79:0/2019773248 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7f032c0b8c90 con 0x7f032c0d2d40 2024-04-26T19:52:03.036 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f030c06ca10 msgr2=0x7f030c06eed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f030c06ca10 0x7f030c06eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 msgr2=0x7f032c1f2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 0x7f032c1f2da0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f031c00e8a0 tx=0x7f031c00ed60 comp rx=0 tx=0).stop 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 shutdown_connections 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f030c06ca10 0x7f030c06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f032c1425a0 0x7f032c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f032c146bc0 0x7f032c142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 --2- 172.21.15.79:0/2019773248 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f032c0d2d40 0x7f032c1f2da0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.039 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 >> 172.21.15.79:0/2019773248 conn(0x7f032c0b4500 msgr2=0x7f032c0ab7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 shutdown_connections 2024-04-26T19:52:03.040 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.038+0000 7f03321a5640 1 -- 172.21.15.79:0/2019773248 wait complete. 2024-04-26T19:52:03.046 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:03.046+0000 7fc7ce920040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T19:52:03.049 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T19:52:03.099 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:03.093+0000 7f7213dce040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:52:03.162 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:03.161+0000 7fc7ce920040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T19:52:03.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.188+0000 7f28412c5640 1 Processor -- start 2024-04-26T19:52:03.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.188+0000 7f28412c5640 1 -- start start 2024-04-26T19:52:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.189+0000 7f28412c5640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 0x7f283c161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.189+0000 7f28412c5640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f283c1634e0 con 0x7f283c0b7ce0 2024-04-26T19:52:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.189+0000 7f28412c5640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f283c1637c0 con 0x7f283c162970 2024-04-26T19:52:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.189+0000 7f28412c5640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f283c164060 con 0x7f283c0b8170 2024-04-26T19:52:03.193 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.189+0000 7f2833fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f283c0b7ce0 0x7f283c161000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59746/0 (socket says 172.21.15.79:59746) 2024-04-26T19:52:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.189+0000 7f2833fff640 1 -- 172.21.15.79:0/3690300357 learned_addr learned my addr 172.21.15.79:0/3690300357 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283b7fe640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 0x7f283c161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1358995338 0 0) 0x7f283c1634e0 con 0x7f283c0b7ce0 2024-04-26T19:52:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2828003520 con 0x7f283c0b7ce0 2024-04-26T19:52:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3131398115 0 0) 0x7f283c1637c0 con 0x7f283c162970 2024-04-26T19:52:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f283c1634e0 con 0x7f283c162970 2024-04-26T19:52:03.194 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4162730066 0 0) 0x7f2828003520 con 0x7f283c0b7ce0 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 msgr2=0x7f283c161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 0x7f283c161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 >> v1:172.21.15.83:6789/0 conn(0x7f283c162970 legacy=0x7f283c160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c163aa0 con 0x7f283c0b7ce0 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f2824001670 con 0x7f283c0b7ce0 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283b7fe640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 0x7f283c161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 >> v1:172.21.15.79:6789/0 conn(0x7f283c0b7ce0 legacy=0x7f283c161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2828003b70 0x7f2828003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f2828004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 0x7f28280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f283c164060 con 0x7f2828004800 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f282800b9f0 con 0x7f2828003b70 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f282800bbf0 con 0x7f283c0b8170 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f2833fff640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f2828004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.195 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283affd640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2828003b70 0x7f2828003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f283b7fe640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 0x7f28280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f2833fff640 1 -- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 msgr2=0x7f28280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f2833fff640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 0x7f28280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f2833fff640 1 -- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2828003b70 msgr2=0x7f2828003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f2833fff640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2828003b70 0x7f2828003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.190+0000 7f2833fff640 1 -- 172.21.15.79:0/3690300357 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2828003520 con 0x7f2828004800 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.191+0000 7f283b7fe640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c0b8170 0x7f28280051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.191+0000 7f2833fff640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f2828004c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2824001eb0 tx=0x7f2828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b4ec9827607e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824007ca0 con 0x7f2828004800 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2824005af0 con 0x7f2828004800 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f283a7fc640 1 -- 172.21.15.79:0/3690300357 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824007070 con 0x7f2828004800 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f28412c5640 1 -- 172.21.15.79:0/3690300357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 msgr2=0x7f2828004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f28412c5640 1 --2- 172.21.15.79:0/3690300357 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f2828004c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2824001eb0 tx=0x7f2828005830 comp rx=0 tx=0).stop 2024-04-26T19:52:03.196 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f283b7fe640 1 -- 172.21.15.79:0/3690300357 reap_dead start 2024-04-26T19:52:03.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f28412c5640 1 -- 172.21.15.79:0/3690300357 shutdown_connections 2024-04-26T19:52:03.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f28412c5640 1 -- 172.21.15.79:0/3690300357 >> 172.21.15.79:0/3690300357 conn(0x7f283c0741a0 msgr2=0x7f283c0b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:03.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f28412c5640 1 -- 172.21.15.79:0/3690300357 shutdown_connections 2024-04-26T19:52:03.197 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.192+0000 7f28412c5640 1 -- 172.21.15.79:0/3690300357 wait complete. 2024-04-26T19:52:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f28412c5640 1 Processor -- start 2024-04-26T19:52:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f28412c5640 1 -- start start 2024-04-26T19:52:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f28412c5640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f283c0ab610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f28412c5640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c15cc50 0x7f283c14ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f28412c5640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c15b410 0x7f283c0cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f28412c5640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f283c0b85a0 con 0x7f2828004800 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f28412c5640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f283c06dc10 con 0x7f283c15cc50 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f28412c5640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f283c162d50 con 0x7f283c15b410 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f283affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f283c0ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f283affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f283c0ab610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54656/0 (socket says 172.21.15.79:54656) 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f283affd640 1 -- 172.21.15.79:0/2630688638 learned_addr learned my addr 172.21.15.79:0/2630688638 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.194+0000 7f2833fff640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c15cc50 0x7f283c14ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f283b7fe640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c15b410 0x7f283c0cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f283affd640 1 -- 172.21.15.79:0/2630688638 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c15b410 msgr2=0x7f283c0cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f283affd640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c15b410 0x7f283c0cdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f283affd640 1 -- 172.21.15.79:0/2630688638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c15cc50 msgr2=0x7f283c14ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f283affd640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c15cc50 0x7f283c14ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f283affd640 1 -- 172.21.15.79:0/2630688638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283c15bbb0 con 0x7f2828004800 2024-04-26T19:52:03.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f283affd640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f283c0ab610 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f282c00f7a0 tx=0x7f282c00fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f28337fe640 1 -- 172.21.15.79:0/2630688638 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c01a070 con 0x7f2828004800 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f28337fe640 1 -- 172.21.15.79:0/2630688638 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f282c005110 con 0x7f2828004800 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283c15be90 con 0x7f2828004800 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f28337fe640 1 -- 172.21.15.79:0/2630688638 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282c007050 con 0x7f2828004800 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.195+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f283c15c450 con 0x7f2828004800 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.196+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f283c0506c0 con 0x7f2828004800 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.196+0000 7f28337fe640 1 -- 172.21.15.79:0/2630688638 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7f282c01d070 con 0x7f2828004800 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.197+0000 7f28337fe640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f281c06ca40 0x7f281c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.197+0000 7f2833fff640 1 -- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f281c06ca40 msgr2=0x7f281c06ef00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.197+0000 7f28337fe640 1 -- 172.21.15.79:0/2630688638 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f282c08b570 con 0x7f2828004800 2024-04-26T19:52:03.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.197+0000 7f2833fff640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f281c06ca40 0x7f281c06ef00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:03.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.201+0000 7f28337fe640 1 -- 172.21.15.79:0/2630688638 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f282c059040 con 0x7f2828004800 2024-04-26T19:52:03.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.397+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f283c06d430 con 0x7f2828004800 2024-04-26T19:52:03.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.397+0000 7f28337fe640 1 -- 172.21.15.79:0/2630688638 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v40) ==== 80+0+352 (secure 0 0 0) 0x7f282c05d340 con 0x7f2828004800 2024-04-26T19:52:03.398 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/16732380 2024-04-27T19:51:58.575815+0000 2024-04-26T19:52:03.399 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/1084811462 2024-04-27T19:51:58.575815+0000 2024-04-26T19:52:03.399 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/3742475428 2024-04-27T19:51:58.575815+0000 2024-04-26T19:52:03.399 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:0/1948040587 2024-04-27T19:51:58.575815+0000 2024-04-26T19:52:03.399 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6833/3161939478 2024-04-27T19:51:58.575815+0000 2024-04-26T19:52:03.399 INFO:teuthology.orchestra.run.smithi079.stdout:172.21.15.79:6832/3161939478 2024-04-27T19:51:58.575815+0000 2024-04-26T19:52:03.399 INFO:teuthology.orchestra.run.smithi079.stderr:listed 6 entries 2024-04-26T19:52:03.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.398+0000 7f2833fff640 1 -- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f281c06ca40 msgr2=0x7f281c06ef00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:03.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.398+0000 7f2833fff640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f281c06ca40 0x7f281c06ef00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T19:52:03.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f281c06ca40 msgr2=0x7f281c06ef00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:03.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f281c06ca40 0x7f281c06ef00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 msgr2=0x7f283c0ab610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f283c0ab610 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f282c00f7a0 tx=0x7f282c00fba0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 shutdown_connections 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f281c06ca40 0x7f281c06ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f283c15b410 0x7f283c0cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f283c15cc50 0x7f283c14ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 --2- 172.21.15.79:0/2630688638 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2828004800 0x7f283c0ab610 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 >> 172.21.15.79:0/2630688638 conn(0x7f283c0741a0 msgr2=0x7f283c0d1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 shutdown_connections 2024-04-26T19:52:03.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.400+0000 7f28412c5640 1 -- 172.21.15.79:0/2630688638 wait complete. 2024-04-26T19:52:03.411 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:03.410+0000 7f7213dce040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:52:03.412 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T19:52:03.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:03.412 DEBUG:teuthology.orchestra.run.smithi079:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T19:52:03.439 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:52:03.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:03.439 DEBUG:teuthology.orchestra.run.smithi079:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T19:52:03.494 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T19:52:03.494 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:52:03.494 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:52:03.496 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:03.496+0000 7fc7ce920040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T19:52:03.510 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056107,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T19:52:03.510 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T19:52:03.510 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:52:03.510 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:52:03.525 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056106,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17615 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T19:52:03.526 INFO:tasks.cephfs.mount:Terminating background process 2024-04-26T19:52:03.526 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:52:03.526 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:52:03.565 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056107,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T19:52:03.565 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:52:03.626 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:03.625+0000 7f7213dce040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:52:03.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.664+0000 7f8094293640 1 Processor -- start 2024-04-26T19:52:03.664 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.664+0000 7f8094293640 1 -- start start 2024-04-26T19:52:03.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.664+0000 7f8094293640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f808c06fb20 0x7f808c0d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.664+0000 7f8094293640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c148fd0 0x7f808c0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f8094293640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f808c0b1290 con 0x7f808c06ff00 2024-04-26T19:52:03.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f8094293640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f808c0d3e60 con 0x7f808c06fb20 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f8094293640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f808c0d4090 con 0x7f808c148fd0 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f809182e640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f808c06ff00 0x7f808c1488c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59762/0 (socket says 172.21.15.79:59762) 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f809182e640 1 -- 172.21.15.79:0/3292984059 learned_addr learned my addr 172.21.15.79:0/3292984059 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f809202f640 1 --2- 172.21.15.79:0/3292984059 >> v2:172.21.15.83:3300/0 conn(0x7f808c06fb20 0x7f808c0d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f8092830640 1 --2- 172.21.15.79:0/3292984059 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c148fd0 0x7f808c0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f809102d640 1 -- 172.21.15.79:0/3292984059 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1481729809 0 0) 0x7f808c0b1290 con 0x7f808c06ff00 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f809102d640 1 -- 172.21.15.79:0/3292984059 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8078003520 con 0x7f808c06ff00 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f8092830640 1 -- 172.21.15.79:0/3292984059 >> v2:172.21.15.83:3300/0 conn(0x7f808c06fb20 msgr2=0x7f808c0d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f8092830640 1 --2- 172.21.15.79:0/3292984059 >> v2:172.21.15.83:3300/0 conn(0x7f808c06fb20 0x7f808c0d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f8092830640 1 -- 172.21.15.79:0/3292984059 >> v1:172.21.15.79:6789/0 conn(0x7f808c06ff00 legacy=0x7f808c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f8092830640 1 -- 172.21.15.79:0/3292984059 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c0d4370 con 0x7f808c148fd0 2024-04-26T19:52:03.667 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.665+0000 7f809202f640 1 --2- 172.21.15.79:0/3292984059 >> v2:172.21.15.83:3300/0 conn(0x7f808c06fb20 0x7f808c0d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:03.671 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.666+0000 7f8092830640 1 --2- 172.21.15.79:0/3292984059 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c148fd0 0x7f808c0d3920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8084009dc0 tx=0x7f808402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8578d40f699d1a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.666+0000 7f809102d640 1 -- 172.21.15.79:0/3292984059 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084007740 con 0x7f808c148fd0 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.666+0000 7f809102d640 1 -- 172.21.15.79:0/3292984059 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80840078e0 con 0x7f808c148fd0 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.666+0000 7f809102d640 1 -- 172.21.15.79:0/3292984059 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084007c90 con 0x7f808c148fd0 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.667+0000 7f8094293640 1 -- 172.21.15.79:0/3292984059 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c148fd0 msgr2=0x7f808c0d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.667+0000 7f8094293640 1 --2- 172.21.15.79:0/3292984059 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c148fd0 0x7f808c0d3920 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8084009dc0 tx=0x7f808402a710 comp rx=0 tx=0).stop 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.667+0000 7f8094293640 1 -- 172.21.15.79:0/3292984059 shutdown_connections 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.667+0000 7f8094293640 1 --2- 172.21.15.79:0/3292984059 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c148fd0 0x7f808c0d3920 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.667+0000 7f8094293640 1 --2- 172.21.15.79:0/3292984059 >> v2:172.21.15.83:3300/0 conn(0x7f808c06fb20 0x7f808c0d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.667+0000 7f8094293640 1 -- 172.21.15.79:0/3292984059 >> 172.21.15.79:0/3292984059 conn(0x7f808c071c30 msgr2=0x7f808c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.667+0000 7f8094293640 1 -- 172.21.15.79:0/3292984059 shutdown_connections 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.667+0000 7f8094293640 1 -- 172.21.15.79:0/3292984059 wait complete. 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.668+0000 7f8094293640 1 Processor -- start 2024-04-26T19:52:03.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.668+0000 7f8094293640 1 -- start start 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f8094293640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c06fb20 0x7f808c1f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f8094293640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808c06ff00 0x7f808c145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f8094293640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808c148fd0 0x7f808c1458b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f8094293640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0d4c10 con 0x7f808c06ff00 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f8094293640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0d4930 con 0x7f808c148fd0 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f8094293640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f808c0aebb0 con 0x7f808c06fb20 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f809182e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808c06ff00 0x7f808c145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f809182e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808c06ff00 0x7f808c145370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54662/0 (socket says 172.21.15.79:54662) 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f809182e640 1 -- 172.21.15.79:0/3852839398 learned_addr learned my addr 172.21.15.79:0/3852839398 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f809202f640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c06fb20 0x7f808c1f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.673 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f8092830640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808c148fd0 0x7f808c1458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f809182e640 1 -- 172.21.15.79:0/3852839398 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c06fb20 msgr2=0x7f808c1f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f809182e640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c06fb20 0x7f808c1f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f809182e640 1 -- 172.21.15.79:0/3852839398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808c148fd0 msgr2=0x7f808c1458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.669+0000 7f809182e640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808c148fd0 0x7f808c1458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.670+0000 7f809182e640 1 -- 172.21.15.79:0/3852839398 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c0d1f50 con 0x7f808c06ff00 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.670+0000 7f809202f640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c06fb20 0x7f808c1f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.670+0000 7f809182e640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808c06ff00 0x7f808c145370 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f807c00c430 tx=0x7f807c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.670+0000 7f8082ffd640 1 -- 172.21.15.79:0/3852839398 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c017070 con 0x7f808c06ff00 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.670+0000 7f8082ffd640 1 -- 172.21.15.79:0/3852839398 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f807c005ce0 con 0x7f808c06ff00 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.670+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c1f5bf0 con 0x7f808c06ff00 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.670+0000 7f8082ffd640 1 -- 172.21.15.79:0/3852839398 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c0072f0 con 0x7f808c06ff00 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.670+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c161000 con 0x7f808c06ff00 2024-04-26T19:52:03.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.671+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c0506c0 con 0x7f808c06ff00 2024-04-26T19:52:03.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.671+0000 7f8082ffd640 1 -- 172.21.15.79:0/3852839398 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7f807c01a070 con 0x7f808c06ff00 2024-04-26T19:52:03.677 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:03.674+0000 7f7213dce040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:52:03.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.672+0000 7f8082ffd640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f807006ca10 0x7f807006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:03.678 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.672+0000 7f809202f640 1 -- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f807006ca10 msgr2=0x7f807006eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:03.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.672+0000 7f8082ffd640 1 -- 172.21.15.79:0/3852839398 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f807c08b3c0 con 0x7f808c06ff00 2024-04-26T19:52:03.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.672+0000 7f809202f640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f807006ca10 0x7f807006eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:03.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.676+0000 7f8082ffd640 1 -- 172.21.15.79:0/3852839398 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f807c058e00 con 0x7f808c06ff00 2024-04-26T19:52:03.717 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:03.716+0000 7fc7ce920040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T19:52:03.768 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:03.767+0000 7fc7ce920040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T19:52:03.788 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:03.787+0000 7f7213dce040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:52:03.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.873+0000 7f809202f640 1 -- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f807006ca10 msgr2=0x7f807006eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:03.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.873+0000 7f809202f640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f807006ca10 0x7f807006eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T19:52:03.885 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:03.884+0000 7fc7ce920040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T19:52:03.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.895+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f808c0d3700 con 0x7f808c06ff00 2024-04-26T19:52:03.898 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.895+0000 7f8082ffd640 1 -- 172.21.15.79:0/3852839398 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f807c05d100 con 0x7f808c06ff00 2024-04-26T19:52:03.898 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:03.899 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2061494256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":2061494256},{"type":"v1","addr":"172.21.15.83:6835","nonce":2061494256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/4034640420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":4034640420},{"type":"v1","addr":"172.21.15.83:6834","nonce":4034640420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/238749789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":238749789},{"type":"v1","addr":"172.21.15.79:6836","nonce":238749789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:51:43.380019+0000","modified":"2024-04-26T19:51:45.374734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/2334214524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":2334214524},{"type":"v1","addr":"172.21.15.79:6837","nonce":2334214524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5343,"qdb_cluster":[5343]},"id":2}]} 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f807006ca10 msgr2=0x7f807006eed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f807006ca10 0x7f807006eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808c06ff00 msgr2=0x7f808c145370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808c06ff00 0x7f808c145370 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f807c00c430 tx=0x7f807c00c8f0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 shutdown_connections 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f807006ca10 0x7f807006eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f808c148fd0 0x7f808c1458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f808c06ff00 0x7f808c145370 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 --2- 172.21.15.79:0/3852839398 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f808c06fb20 0x7f808c1f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:03.900 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 >> 172.21.15.79:0/3852839398 conn(0x7f808c071c30 msgr2=0x7f808c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:03.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 shutdown_connections 2024-04-26T19:52:03.901 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:03.897+0000 7f8094293640 1 -- 172.21.15.79:0/3852839398 wait complete. 2024-04-26T19:52:03.909 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:52:04.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.007+0000 7fe4c63a4640 1 Processor -- start 2024-04-26T19:52:04.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.007+0000 7fe4c63a4640 1 -- start start 2024-04-26T19:52:04.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4c63a4640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4c0162970 0x7fe4c0160440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4c63a4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 0x7fe4c00b8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4c63a4640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4c01634e0 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4c63a4640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fe4c0164060 con 0x7fe4c0162970 2024-04-26T19:52:04.009 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4c63a4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c01637c0 con 0x7fe4c0162010 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4bffff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fe4c0162970 0x7fe4c0160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4c4941640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 0x7fe4c00b8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4bf7fe640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7fe4c00b7ce0 0x7fe4c006c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:58614/0 (socket says 172.21.15.79:58614) 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4bf7fe640 1 -- 172.21.15.79:0/686206734 learned_addr learned my addr 172.21.15.79:0/686206734 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4bffff640 1 -- 172.21.15.79:0/686206734 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 msgr2=0x7fe4c00b8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4bffff640 1 --2- 172.21.15.79:0/686206734 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 0x7fe4c00b8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4bffff640 1 -- 172.21.15.79:0/686206734 >> v1:172.21.15.83:6789/0 conn(0x7fe4c00b7ce0 legacy=0x7fe4c006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4bffff640 1 -- 172.21.15.79:0/686206734 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c0163aa0 con 0x7fe4c0162970 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4beffd640 1 -- 172.21.15.79:0/686206734 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3564614563 0 0) 0x7fe4c01634e0 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.008+0000 7fe4c4941640 1 --2- 172.21.15.79:0/686206734 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 0x7fe4c00b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.009+0000 7fe4bffff640 1 --2- 172.21.15.79:0/686206734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c0162970 0x7fe4c0160440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40097a0 tx=0x7fe4b402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c493b72ce972127d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.009+0000 7fe4beffd640 1 -- 172.21.15.79:0/686206734 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4037070 con 0x7fe4c0162970 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.009+0000 7fe4beffd640 1 -- 172.21.15.79:0/686206734 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b402ae50 con 0x7fe4c0162970 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.009+0000 7fe4beffd640 1 -- 172.21.15.79:0/686206734 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4032660 con 0x7fe4c0162970 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.010+0000 7fe4c63a4640 1 -- 172.21.15.79:0/686206734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c0162970 msgr2=0x7fe4c0160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.010+0000 7fe4c63a4640 1 --2- 172.21.15.79:0/686206734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c0162970 0x7fe4c0160440 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40097a0 tx=0x7fe4b402a100 comp rx=0 tx=0).stop 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.010+0000 7fe4c63a4640 1 -- 172.21.15.79:0/686206734 shutdown_connections 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.010+0000 7fe4c63a4640 1 --2- 172.21.15.79:0/686206734 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 0x7fe4c00b8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.010+0000 7fe4c63a4640 1 --2- 172.21.15.79:0/686206734 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c0162970 0x7fe4c0160440 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.010+0000 7fe4c63a4640 1 -- 172.21.15.79:0/686206734 >> 172.21.15.79:0/686206734 conn(0x7fe4c00741a0 msgr2=0x7fe4c00b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.010+0000 7fe4c63a4640 1 -- 172.21.15.79:0/686206734 shutdown_connections 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.010+0000 7fe4c63a4640 1 -- 172.21.15.79:0/686206734 wait complete. 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.011+0000 7fe4c63a4640 1 Processor -- start 2024-04-26T19:52:04.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.011+0000 7fe4c63a4640 1 -- start start 2024-04-26T19:52:04.014 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4c63a4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4c00b7ce0 0x7fe4c01f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4c63a4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 0x7fe4c015a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4c63a4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c0162970 0x7fe4c01578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4c63a4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0160980 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4c63a4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c006dc10 con 0x7fe4c0162970 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4c63a4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c0162d50 con 0x7fe4c0162010 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4c00b7ce0 0x7fe4c01f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4c00b7ce0 0x7fe4c01f57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54670/0 (socket says 172.21.15.79:54670) 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4bffff640 1 -- 172.21.15.79:0/3919717772 learned_addr learned my addr 172.21.15.79:0/3919717772 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4c4941640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c0162970 0x7fe4c01578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4bf7fe640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 0x7fe4c015a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4bffff640 1 -- 172.21.15.79:0/3919717772 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 msgr2=0x7fe4c015a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4bffff640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 0x7fe4c015a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.015 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4bffff640 1 -- 172.21.15.79:0/3919717772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c0162970 msgr2=0x7fe4c01578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4bffff640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c0162970 0x7fe4c01578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.012+0000 7fe4bffff640 1 -- 172.21.15.79:0/3919717772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c015ac70 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.013+0000 7fe4bffff640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4c00b7ce0 0x7fe4c01f57b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40097a0 tx=0x7fe4b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:04.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.013+0000 7fe4bcff9640 1 -- 172.21.15.79:0/3919717772 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4037040 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.013+0000 7fe4bcff9640 1 -- 172.21.15.79:0/3919717772 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4b4004030 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.013+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c01f4d10 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.013+0000 7fe4bcff9640 1 -- 172.21.15.79:0/3919717772 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4006e10 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.013+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c01f4fb0 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.014+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c01f4b20 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.014+0000 7fe4bcff9640 1 -- 172.21.15.79:0/3919717772 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7fe4b4003610 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.015+0000 7fe4bcff9640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fe4ac06ca10 0x7fe4ac06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.015+0000 7fe4bf7fe640 1 -- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fe4ac06ca10 msgr2=0x7fe4ac06eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:04.017 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.015+0000 7fe4bcff9640 1 -- 172.21.15.79:0/3919717772 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fe4b40acb90 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.015+0000 7fe4bf7fe640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fe4ac06ca10 0x7fe4ac06eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:04.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.019+0000 7fe4bcff9640 1 -- 172.21.15.79:0/3919717772 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fe4b407ac20 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.058 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:04.057+0000 7f7213dce040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:52:04.101 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:04.100+0000 7f7213dce040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:52:04.170 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:04.169+0000 7fc7ce920040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T19:52:04.204 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.204+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe4c006d430 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.205+0000 7fe4bcff9640 1 -- 172.21.15.79:0/3919717772 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) ==== 74+0+17269 (secure 0 0 0) 0x7fe4b407a520 con 0x7fe4c00b7ce0 2024-04-26T19:52:04.206 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:04.206 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":40,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:51:58.575840+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:51:41.583741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T19:51:42.339687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:0/16732380":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1084811462":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/3742475428":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1948040587":"2024-04-27T19:51:58.575815+0000","172.21.15.79:6833/3161939478":"2024-04-27T19:51:58.575815+0000","172.21.15.79:6832/3161939478":"2024-04-27T19:51:58.575815+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:52:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fe4ac06ca10 msgr2=0x7fe4ac06eed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fe4ac06ca10 0x7fe4ac06eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4c00b7ce0 msgr2=0x7fe4c01f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4c00b7ce0 0x7fe4c01f57b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40097a0 tx=0x7fe4b40046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 shutdown_connections 2024-04-26T19:52:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fe4ac06ca10 0x7fe4ac06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.209 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fe4c0162970 0x7fe4c01578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fe4c0162010 0x7fe4c015a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 --2- 172.21.15.79:0/3919717772 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fe4c00b7ce0 0x7fe4c01f57b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 >> 172.21.15.79:0/3919717772 conn(0x7fe4c00741a0 msgr2=0x7fe4c00b8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 shutdown_connections 2024-04-26T19:52:04.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.208+0000 7fe4c63a4640 1 -- 172.21.15.79:0/3919717772 wait complete. 2024-04-26T19:52:04.220 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:04.219+0000 7fc7ce920040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T19:52:04.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T19:50:18.265547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T19:51:41.583741+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T19:51:42.339687+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T19:52:04.224 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T19:52:04.242 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:04.241+0000 7f7213dce040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:52:04.329 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.325+0000 7f4f10c08640 1 Processor -- start 2024-04-26T19:52:04.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.325+0000 7f4f10c08640 1 -- start start 2024-04-26T19:52:04.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f10c08640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c15c2e0 0x7f4f0c06f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f10c08640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f0c0cee40 con 0x7f4f0c073730 2024-04-26T19:52:04.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f10c08640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f0c0cf120 con 0x7f4f0c15bf00 2024-04-26T19:52:04.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f10c08640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0c074880 con 0x7f4f0c15c2e0 2024-04-26T19:52:04.330 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f0ad9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f4f0c073730 0x7f4f0c0cfee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59770/0 (socket says 172.21.15.79:59770) 2024-04-26T19:52:04.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f0ad9d640 1 -- 172.21.15.79:0/2396599955 learned_addr learned my addr 172.21.15.79:0/2396599955 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:04.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f03fff640 1 --2- 172.21.15.79:0/2396599955 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c15c2e0 0x7f4f0c06f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f09d9b640 1 -- 172.21.15.79:0/2396599955 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3217528755 0 0) 0x7f4f0c0cee40 con 0x7f4f0c073730 2024-04-26T19:52:04.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f09d9b640 1 -- 172.21.15.79:0/2396599955 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ef8003520 con 0x7f4f0c073730 2024-04-26T19:52:04.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f09d9b640 1 -- 172.21.15.79:0/2396599955 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1566678434 0 0) 0x7f4f0c0cf120 con 0x7f4f0c15bf00 2024-04-26T19:52:04.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f09d9b640 1 -- 172.21.15.79:0/2396599955 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f0c0cee40 con 0x7f4f0c15bf00 2024-04-26T19:52:04.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f03fff640 1 -- 172.21.15.79:0/2396599955 >> v1:172.21.15.83:6789/0 conn(0x7f4f0c15bf00 legacy=0x7f4f0c074170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f03fff640 1 -- 172.21.15.79:0/2396599955 >> v1:172.21.15.79:6789/0 conn(0x7f4f0c073730 legacy=0x7f4f0c0cfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.326+0000 7f4f03fff640 1 -- 172.21.15.79:0/2396599955 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c0cf400 con 0x7f4f0c15c2e0 2024-04-26T19:52:04.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.327+0000 7f4f03fff640 1 --2- 172.21.15.79:0/2396599955 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c15c2e0 0x7f4f0c06f0e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097d0 tx=0x7f4ef402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d8a0aa94d5c8382 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:04.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.327+0000 7f4f09d9b640 1 -- 172.21.15.79:0/2396599955 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4037070 con 0x7f4f0c15c2e0 2024-04-26T19:52:04.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.327+0000 7f4f09d9b640 1 -- 172.21.15.79:0/2396599955 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef4032440 con 0x7f4f0c15c2e0 2024-04-26T19:52:04.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.327+0000 7f4f09d9b640 1 -- 172.21.15.79:0/2396599955 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40327f0 con 0x7f4f0c15c2e0 2024-04-26T19:52:04.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.328+0000 7f4f10c08640 1 -- 172.21.15.79:0/2396599955 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c15c2e0 msgr2=0x7f4f0c06f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.328+0000 7f4f10c08640 1 --2- 172.21.15.79:0/2396599955 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c15c2e0 0x7f4f0c06f0e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4ef40097d0 tx=0x7f4ef402a180 comp rx=0 tx=0).stop 2024-04-26T19:52:04.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.328+0000 7f4f10c08640 1 -- 172.21.15.79:0/2396599955 shutdown_connections 2024-04-26T19:52:04.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.328+0000 7f4f10c08640 1 --2- 172.21.15.79:0/2396599955 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c15c2e0 0x7f4f0c06f0e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.328+0000 7f4f10c08640 1 -- 172.21.15.79:0/2396599955 >> 172.21.15.79:0/2396599955 conn(0x7f4f0c068a90 msgr2=0x7f4f0c0b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:04.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.328+0000 7f4f10c08640 1 -- 172.21.15.79:0/2396599955 shutdown_connections 2024-04-26T19:52:04.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.328+0000 7f4f10c08640 1 -- 172.21.15.79:0/2396599955 wait complete. 2024-04-26T19:52:04.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.329+0000 7f4f10c08640 1 Processor -- start 2024-04-26T19:52:04.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.329+0000 7f4f10c08640 1 -- start start 2024-04-26T19:52:04.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f10c08640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c073730 0x7f4f0c155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f10c08640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f0c15bf00 0x7f4f0c1f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f10c08640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0c15c2e0 0x7f4f0c153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f10c08640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c06f620 con 0x7f4f0c15bf00 2024-04-26T19:52:04.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f10c08640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c0cf9c0 con 0x7f4f0c15c2e0 2024-04-26T19:52:04.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f10c08640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0c06f7a0 con 0x7f4f0c073730 2024-04-26T19:52:04.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f0a59c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c073730 0x7f4f0c155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f03fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f0c15bf00 0x7f4f0c1f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f0ad9d640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0c15c2e0 0x7f4f0c153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f03fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f0c15bf00 0x7f4f0c1f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54676/0 (socket says 172.21.15.79:54676) 2024-04-26T19:52:04.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f0a59c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c073730 0x7f4f0c155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:48552/0 (socket says 172.21.15.79:48552) 2024-04-26T19:52:04.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.330+0000 7f4f03fff640 1 -- 172.21.15.79:0/1795934787 learned_addr learned my addr 172.21.15.79:0/1795934787 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:04.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f0a59c640 1 -- 172.21.15.79:0/1795934787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0c15c2e0 msgr2=0x7f4f0c153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f0a59c640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0c15c2e0 0x7f4f0c153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f0a59c640 1 -- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f0c15bf00 msgr2=0x7f4f0c1f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f0a59c640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f0c15bf00 0x7f4f0c1f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f0a59c640 1 -- 172.21.15.79:0/1795934787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c1543e0 con 0x7f4f0c073730 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f03fff640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f0c15bf00 0x7f4f0c1f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f0a59c640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c073730 0x7f4f0c155430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00c3c0 tx=0x7f4efc00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f02ffd640 1 -- 172.21.15.79:0/1795934787 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc017070 con 0x7f4f0c073730 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f02ffd640 1 -- 172.21.15.79:0/1795934787 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4efc0052f0 con 0x7f4f0c073730 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c154970 con 0x7f4f0c073730 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f02ffd640 1 -- 172.21.15.79:0/1795934787 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4efc003830 con 0x7f4f0c073730 2024-04-26T19:52:04.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.331+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c154f30 con 0x7f4f0c073730 2024-04-26T19:52:04.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.332+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0c0506c0 con 0x7f4f0c073730 2024-04-26T19:52:04.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.333+0000 7f4f02ffd640 1 -- 172.21.15.79:0/1795934787 <== mon.2 v2:172.21.15.83:3301/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7f4efc01a070 con 0x7f4f0c073730 2024-04-26T19:52:04.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.333+0000 7f4f02ffd640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4eec06ca10 0x7f4eec06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.333+0000 7f4f03fff640 1 -- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4eec06ca10 msgr2=0x7f4eec06eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:04.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.333+0000 7f4f02ffd640 1 -- 172.21.15.79:0/1795934787 <== mon.2 v2:172.21.15.83:3301/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7f4efc08b2b0 con 0x7f4f0c073730 2024-04-26T19:52:04.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.333+0000 7f4f03fff640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4eec06ca10 0x7f4eec06eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:04.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.342+0000 7f4f02ffd640 1 -- 172.21.15.79:0/1795934787 <== mon.2 v2:172.21.15.83:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4efc058cd0 con 0x7f4f0c073730 2024-04-26T19:52:04.376 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:04.371+0000 7fc7ce920040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T19:52:04.377 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:04.376+0000 7f7213dce040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:52:04.425 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:04.424+0000 7f7213dce040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:52:04.477 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:04.476+0000 7f7213dce040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:52:04.508 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:04.508+0000 7fc7ce920040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T19:52:04.519 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:04.518+0000 7f7213dce040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:52:04.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.522+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4f0c158130 con 0x7f4f0c073730 2024-04-26T19:52:04.523 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.523+0000 7f4f02ffd640 1 -- 172.21.15.79:0/1795934787 <== mon.2 v2:172.21.15.83:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) ==== 93+0+10 (secure 0 0 0) 0x7f4efc05cfd0 con 0x7f4f0c073730 2024-04-26T19:52:04.524 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 1 2024-04-26T19:52:04.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4eec06ca10 msgr2=0x7f4eec06eed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:04.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4eec06ca10 0x7f4eec06eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c073730 msgr2=0x7f4f0c155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c073730 0x7f4f0c155430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00c3c0 tx=0x7f4efc00c880 comp rx=0 tx=0).stop 2024-04-26T19:52:04.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 shutdown_connections 2024-04-26T19:52:04.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7f4eec06ca10 0x7f4eec06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4f0c15c2e0 0x7f4f0c153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4f0c15bf00 0x7f4f0c1f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 --2- 172.21.15.79:0/1795934787 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4f0c073730 0x7f4f0c155430 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 >> 172.21.15.79:0/1795934787 conn(0x7f4f0c068a90 msgr2=0x7f4f0c1604a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:04.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 shutdown_connections 2024-04-26T19:52:04.527 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.525+0000 7f4f10c08640 1 -- 172.21.15.79:0/1795934787 wait complete. 2024-04-26T19:52:04.537 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T19:52:04.560 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:04.559+0000 7fc7ce920040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T19:52:04.617 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:04.616+0000 7fc7ce920040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T19:52:04.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.638+0000 7eff9e415640 1 Processor -- start 2024-04-26T19:52:04.639 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.638+0000 7eff9e415640 1 -- start start 2024-04-26T19:52:04.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff9e415640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff980b3a30 0x7eff981631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff9e415640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff98163700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff9e415640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff9806c3f0 con 0x7eff98142b60 2024-04-26T19:52:04.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff9e415640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7eff981428e0 con 0x7eff980b3a30 2024-04-26T19:52:04.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff9e415640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7eff9806ccb0 con 0x7eff980b3e10 2024-04-26T19:52:04.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff97fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff980b3a30 0x7eff981631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff977fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff98163700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff9c9b2640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7eff98142b60 0x7eff98142fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:58622/0 (socket says 172.21.15.79:58622) 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff97fff640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7eff980b3a30 0x7eff981631c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:54272/0 (socket says 172.21.15.79:54272) 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff977fe640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff98163700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:48566/0 (socket says 172.21.15.79:48566) 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.639+0000 7eff9c9b2640 1 -- 172.21.15.79:0/2284156135 learned_addr learned my addr 172.21.15.79:0/2284156135 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.640+0000 7eff96ffd640 1 -- 172.21.15.79:0/2284156135 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2127990556 0 0) 0x7eff9806c3f0 con 0x7eff98142b60 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.640+0000 7eff977fe640 1 -- 172.21.15.79:0/2284156135 >> v2:172.21.15.83:3300/0 conn(0x7eff980b3a30 msgr2=0x7eff981631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.640+0000 7eff977fe640 1 --2- 172.21.15.79:0/2284156135 >> v2:172.21.15.83:3300/0 conn(0x7eff980b3a30 0x7eff981631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.640+0000 7eff977fe640 1 -- 172.21.15.79:0/2284156135 >> v1:172.21.15.83:6789/0 conn(0x7eff98142b60 legacy=0x7eff98142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.640+0000 7eff977fe640 1 -- 172.21.15.79:0/2284156135 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff9806c660 con 0x7eff980b3e10 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.640+0000 7eff97fff640 1 --2- 172.21.15.79:0/2284156135 >> v2:172.21.15.83:3300/0 conn(0x7eff980b3a30 0x7eff981631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.640+0000 7eff977fe640 1 --2- 172.21.15.79:0/2284156135 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff98163700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff9806ce30 tx=0x7eff8802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d35f9159022122 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff96ffd640 1 -- 172.21.15.79:0/2284156135 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88037070 con 0x7eff980b3e10 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff96ffd640 1 -- 172.21.15.79:0/2284156135 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff88032440 con 0x7eff980b3e10 2024-04-26T19:52:04.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff96ffd640 1 -- 172.21.15.79:0/2284156135 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88032810 con 0x7eff980b3e10 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff9e415640 1 -- 172.21.15.79:0/2284156135 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 msgr2=0x7eff98163700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff9e415640 1 --2- 172.21.15.79:0/2284156135 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff98163700 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff9806ce30 tx=0x7eff8802a180 comp rx=0 tx=0).stop 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff9e415640 1 -- 172.21.15.79:0/2284156135 shutdown_connections 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff9e415640 1 --2- 172.21.15.79:0/2284156135 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff98163700 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff9e415640 1 --2- 172.21.15.79:0/2284156135 >> v2:172.21.15.83:3300/0 conn(0x7eff980b3a30 0x7eff981631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff9e415640 1 -- 172.21.15.79:0/2284156135 >> 172.21.15.79:0/2284156135 conn(0x7eff980685d0 msgr2=0x7eff98073b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff9e415640 1 -- 172.21.15.79:0/2284156135 shutdown_connections 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.641+0000 7eff9e415640 1 -- 172.21.15.79:0/2284156135 wait complete. 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.642+0000 7eff9e415640 1 Processor -- start 2024-04-26T19:52:04.646 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff9e415640 1 -- start start 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff9e415640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff980b3a30 0x7eff980b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff9e415640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff9813fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff9e415640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff98142b60 0x7eff9806ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff9e415640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7eff98163f40 con 0x7eff980b3a30 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff9e415640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7eff98163c40 con 0x7eff98142b60 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff9e415640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7eff9806d140 con 0x7eff980b3e10 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff97fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff980b3a30 0x7eff980b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff97fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff980b3a30 0x7eff980b0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54690/0 (socket says 172.21.15.79:54690) 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff97fff640 1 -- 172.21.15.79:0/2235959036 learned_addr learned my addr 172.21.15.79:0/2235959036 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff977fe640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff9813fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.643+0000 7eff9c9b2640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff98142b60 0x7eff9806ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff97fff640 1 -- 172.21.15.79:0/2235959036 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 msgr2=0x7eff9813fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff97fff640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff9813fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff97fff640 1 -- 172.21.15.79:0/2235959036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff98142b60 msgr2=0x7eff9806ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.647 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff97fff640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff98142b60 0x7eff9806ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff97fff640 1 -- 172.21.15.79:0/2235959036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff981e9320 con 0x7eff980b3a30 2024-04-26T19:52:04.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff97fff640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff980b3a30 0x7eff980b0ad0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00e960 tx=0x7eff8c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:04.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff94ff9640 1 -- 172.21.15.79:0/2235959036 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c017070 con 0x7eff980b3a30 2024-04-26T19:52:04.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff94ff9640 1 -- 172.21.15.79:0/2235959036 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff8c00ca00 con 0x7eff980b3a30 2024-04-26T19:52:04.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff981e9600 con 0x7eff980b3a30 2024-04-26T19:52:04.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff94ff9640 1 -- 172.21.15.79:0/2235959036 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00cdd0 con 0x7eff980b3a30 2024-04-26T19:52:04.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.644+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff981e9bc0 con 0x7eff980b3a30 2024-04-26T19:52:04.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.645+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff980b14c0 con 0x7eff980b3a30 2024-04-26T19:52:04.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.645+0000 7eff94ff9640 1 -- 172.21.15.79:0/2235959036 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7eff8c01a070 con 0x7eff980b3a30 2024-04-26T19:52:04.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.646+0000 7eff94ff9640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7eff8406ca10 0x7eff8406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.646+0000 7eff977fe640 1 -- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7eff8406ca10 msgr2=0x7eff8406eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:04.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.646+0000 7eff94ff9640 1 -- 172.21.15.79:0/2235959036 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7eff8c08c160 con 0x7eff980b3a30 2024-04-26T19:52:04.649 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.646+0000 7eff977fe640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7eff8406ca10 0x7eff8406eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:04.655 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.654+0000 7eff94ff9640 1 -- 172.21.15.79:0/2235959036 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7eff980b14c0 con 0x7eff980b3a30 2024-04-26T19:52:04.666 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:04.665+0000 7fc7ce920040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T19:52:04.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.835+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7eff980506c0 con 0x7eff980b3a30 2024-04-26T19:52:04.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.835+0000 7eff94ff9640 1 -- 172.21.15.79:0/2235959036 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) ==== 104+0+11 (secure 0 0 0) 0x7eff8c059bb0 con 0x7eff980b3a30 2024-04-26T19:52:04.836 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 32 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7eff8406ca10 msgr2=0x7eff8406eed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7eff8406ca10 0x7eff8406eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff980b3a30 msgr2=0x7eff980b0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff980b3a30 0x7eff980b0ad0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00e960 tx=0x7eff8c00ee20 comp rx=0 tx=0).stop 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 shutdown_connections 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7eff8406ca10 0x7eff8406eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7eff98142b60 0x7eff9806ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7eff980b3e10 0x7eff9813fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 --2- 172.21.15.79:0/2235959036 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7eff980b3a30 0x7eff980b0ad0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 >> 172.21.15.79:0/2235959036 conn(0x7eff980685d0 msgr2=0x7eff98160c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:04.839 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 shutdown_connections 2024-04-26T19:52:04.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.838+0000 7eff9e415640 1 -- 172.21.15.79:0/2235959036 wait complete. 2024-04-26T19:52:04.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T19:52:04.909 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:04.908+0000 7f7213dce040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:52:04.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.990+0000 7fa6603bf640 1 Processor -- start 2024-04-26T19:52:04.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.990+0000 7fa6603bf640 1 -- start start 2024-04-26T19:52:04.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.991+0000 7fa6603bf640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa65815c020 0x7fa6580aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.991+0000 7fa6603bf640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fa65806f740 0x7fa6580d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:04.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.991+0000 7fa6603bf640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6580aaf40 con 0x7fa65815c400 2024-04-26T19:52:04.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.991+0000 7fa6603bf640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fa6580ab220 con 0x7fa65815c020 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.991+0000 7fa6603bf640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fa6580ab500 con 0x7fa65806f740 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.991+0000 7fa65e15b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa65815c020 0x7fa6580aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.992+0000 7fa65e15b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fa65815c020 0x7fa6580aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54696/0 (socket says 172.21.15.79:54696) 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.992+0000 7fa65e15b640 1 -- 172.21.15.79:0/1400065776 learned_addr learned my addr 172.21.15.79:0/1400065776 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.992+0000 7fa65e95c640 1 --2- 172.21.15.79:0/1400065776 >> v2:172.21.15.83:3300/0 conn(0x7fa65806f740 0x7fa6580d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.992+0000 7fa65e15b640 1 -- 172.21.15.79:0/1400065776 >> v2:172.21.15.83:3300/0 conn(0x7fa65806f740 msgr2=0x7fa6580d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.992+0000 7fa65e15b640 1 --2- 172.21.15.79:0/1400065776 >> v2:172.21.15.83:3300/0 conn(0x7fa65806f740 0x7fa6580d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.992+0000 7fa65e15b640 1 -- 172.21.15.79:0/1400065776 >> v1:172.21.15.83:6789/0 conn(0x7fa65815c400 legacy=0x7fa65806f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.992+0000 7fa65e15b640 1 -- 172.21.15.79:0/1400065776 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6580d08d0 con 0x7fa65815c020 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.992+0000 7fa65e15b640 1 --2- 172.21.15.79:0/1400065776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 0x7fa6580aa4c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa65806d030 tx=0x7fa65402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa02a46bfee986cc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa65d159640 1 -- 172.21.15.79:0/1400065776 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654036020 con 0x7fa65815c020 2024-04-26T19:52:04.995 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa65d159640 1 -- 172.21.15.79:0/1400065776 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa654032640 con 0x7fa65815c020 2024-04-26T19:52:04.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa65d159640 1 -- 172.21.15.79:0/1400065776 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654032a10 con 0x7fa65815c020 2024-04-26T19:52:04.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa6603bf640 1 -- 172.21.15.79:0/1400065776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 msgr2=0x7fa6580aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:04.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa6603bf640 1 --2- 172.21.15.79:0/1400065776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 0x7fa6580aa4c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa65806d030 tx=0x7fa65402c940 comp rx=0 tx=0).stop 2024-04-26T19:52:04.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa6603bf640 1 -- 172.21.15.79:0/1400065776 shutdown_connections 2024-04-26T19:52:04.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa6603bf640 1 --2- 172.21.15.79:0/1400065776 >> v2:172.21.15.83:3300/0 conn(0x7fa65806f740 0x7fa6580d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa6603bf640 1 --2- 172.21.15.79:0/1400065776 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 0x7fa6580aa4c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:04.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa6603bf640 1 -- 172.21.15.79:0/1400065776 >> 172.21.15.79:0/1400065776 conn(0x7fa6580b3ed0 msgr2=0x7fa6580681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:04.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa6603bf640 1 -- 172.21.15.79:0/1400065776 shutdown_connections 2024-04-26T19:52:04.996 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.993+0000 7fa6603bf640 1 -- 172.21.15.79:0/1400065776 wait complete. 2024-04-26T19:52:05.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa6603bf640 1 Processor -- start 2024-04-26T19:52:05.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa6603bf640 1 -- start start 2024-04-26T19:52:05.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa6603bf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa65806f740 0x7fa6580cf580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa6603bf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 0x7fa6581f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa6603bf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa658072990 0x7fa6581f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa6603bf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa6580aab80 con 0x7fa65815c020 2024-04-26T19:52:05.001 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa6603bf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa6580aaa00 con 0x7fa65806f740 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa6603bf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa6580aad00 con 0x7fa658072990 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa65d95a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 0x7fa6581f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa65d95a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 0x7fa6581f1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54698/0 (socket says 172.21.15.79:54698) 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa65e15b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa65806f740 0x7fa6580cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.995+0000 7fa65d95a640 1 -- 172.21.15.79:0/408547409 learned_addr learned my addr 172.21.15.79:0/408547409 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa65e95c640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa658072990 0x7fa6581f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa65d95a640 1 -- 172.21.15.79:0/408547409 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa658072990 msgr2=0x7fa6581f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa65d95a640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa658072990 0x7fa6581f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa65d95a640 1 -- 172.21.15.79:0/408547409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa65806f740 msgr2=0x7fa6580cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa65d95a640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa65806f740 0x7fa6580cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa65d95a640 1 -- 172.21.15.79:0/408547409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6581f1db0 con 0x7fa65815c020 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa65d95a640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 0x7fa6581f1680 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00c3c0 tx=0x7fa64c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:05.002 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa64affd640 1 -- 172.21.15.79:0/408547409 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c0052d0 con 0x7fa65815c020 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa64affd640 1 -- 172.21.15.79:0/408547409 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa64c005490 con 0x7fa65815c020 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa64affd640 1 -- 172.21.15.79:0/408547409 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c007260 con 0x7fa65815c020 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6581f2090 con 0x7fa65815c020 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.996+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa658072e20 con 0x7fa65815c020 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.997+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa658160b20 con 0x7fa65815c020 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.998+0000 7fa64affd640 1 -- 172.21.15.79:0/408547409 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 11) ==== 91143+0+0 (secure 0 0 0) 0x7fa64c007400 con 0x7fa65815c020 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.998+0000 7fa64affd640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fa63c06ca10 0x7fa63c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.998+0000 7fa65e15b640 1 -- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fa63c06ca10 msgr2=0x7fa63c06eed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.79:6832/3922824612 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.998+0000 7fa64affd640 1 -- 172.21.15.79:0/408547409 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6359+0+0 (secure 0 0 0) 0x7fa64c08b470 con 0x7fa65815c020 2024-04-26T19:52:05.003 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:04.998+0000 7fa65e15b640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fa63c06ca10 0x7fa63c06eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T19:52:05.006 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.006+0000 7fa64affd640 1 -- 172.21.15.79:0/408547409 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa658160b20 con 0x7fa65815c020 2024-04-26T19:52:05.024 INFO:tasks.ceph.mgr.x.smithi083.stderr:2024-04-26T19:52:05.023+0000 7f7213dce040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:52:05.076 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.076+0000 7fc7ce920040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T19:52:05.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.188+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa6581f25b0 con 0x7fa65815c020 2024-04-26T19:52:05.189 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.189+0000 7fa64affd640 1 -- 172.21.15.79:0/408547409 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) ==== 100+0+11 (secure 0 0 0) 0x7fa64c058e30 con 0x7fa65815c020 2024-04-26T19:52:05.189 INFO:teuthology.orchestra.run.smithi079.stdout:pg_num: 64 2024-04-26T19:52:05.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.190+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fa63c06ca10 msgr2=0x7fa63c06eed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T19:52:05.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.190+0000 7fa6603bf640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fa63c06ca10 0x7fa63c06eed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.191 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.190+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 msgr2=0x7fa6581f1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.190+0000 7fa6603bf640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 0x7fa6581f1680 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00c3c0 tx=0x7fa64c00c880 comp rx=0 tx=0).stop 2024-04-26T19:52:05.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.191+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 shutdown_connections 2024-04-26T19:52:05.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.191+0000 7fa6603bf640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:6832/3922824612,v1:172.21.15.79:6833/3922824612] conn(0x7fa63c06ca10 0x7fa63c06eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.191+0000 7fa6603bf640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa658072990 0x7fa6581f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.191+0000 7fa6603bf640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa65815c020 0x7fa6581f1680 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.191+0000 7fa6603bf640 1 --2- 172.21.15.79:0/408547409 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa65806f740 0x7fa6580cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.191+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 >> 172.21.15.79:0/408547409 conn(0x7fa6580b3ed0 msgr2=0x7fa658150b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:05.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.191+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 shutdown_connections 2024-04-26T19:52:05.192 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.191+0000 7fa6603bf640 1 -- 172.21.15.79:0/408547409 wait complete. 2024-04-26T19:52:05.196 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.196+0000 7fc7ce920040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T19:52:05.204 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:52:05.254 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc752b20640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.254 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc752b20640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.255 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc752b20640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.255 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc752b20640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.255 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc752b20640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.255 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc74f319640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.255 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc74f319640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.255 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc74f319640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.255 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc74f319640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.255 INFO:tasks.ceph.mgr.y.smithi079.stderr:2024-04-26T19:52:05.253+0000 7fc74f319640 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T19:52:05.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.308+0000 7f37c5815640 1 Processor -- start 2024-04-26T19:52:05.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.308+0000 7f37c5815640 1 -- start start 2024-04-26T19:52:05.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37c5815640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37c00b4220 0x7f37c01630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37c5815640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f37c0160770 0x7f37c0163610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37c5815640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37c0144ca0 con 0x7f37c0160390 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37c5815640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f37c0163b50 con 0x7f37c0160770 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37c5815640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f37c0163e00 con 0x7f37c00b4220 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37beffd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f37c0160390 0x7f37c00b3b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59772/0 (socket says 172.21.15.79:59772) 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37beffd640 1 -- 172.21.15.79:0/2431108886 learned_addr learned my addr 172.21.15.79:0/2431108886 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1495431643 0 0) 0x7f37c0144ca0 con 0x7f37c0160390 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37be7fc640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37c00b4220 0x7f37c01630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37a8003520 con 0x7f37c0160390 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.309+0000 7f37bf7fe640 1 --2- 172.21.15.79:0/2431108886 >> v2:172.21.15.83:3300/0 conn(0x7f37c0160770 0x7f37c0163610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3168509566 0 0) 0x7f37a8003520 con 0x7f37c0160390 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37c00b4220 msgr2=0x7f37c01630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:05.313 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37c00b4220 0x7f37c01630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 >> v2:172.21.15.83:3300/0 conn(0x7f37c0160770 msgr2=0x7f37c0163610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 --2- 172.21.15.79:0/2431108886 >> v2:172.21.15.83:3300/0 conn(0x7f37c0160770 0x7f37c0163610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c0144f30 con 0x7f37c0160390 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f37b4001670 con 0x7f37c0160390 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37be7fc640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37c00b4220 0x7f37c01630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 >> v1:172.21.15.79:6789/0 conn(0x7f37c0160390 legacy=0x7f37c00b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a80039b0 0x7f37a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37c00b4220 0x7f37a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37a8004d40 0x7f37a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0163b50 con 0x7f37a80039b0 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0163e00 con 0x7f37a8004d40 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f37a800b9c0 con 0x7f37c00b4220 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37beffd640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a80039b0 0x7f37a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.314 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37be7fc640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37c00b4220 0x7f37a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37bf7fe640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37a8004d40 0x7f37a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37beffd640 1 -- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37c00b4220 msgr2=0x7f37a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37beffd640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37c00b4220 0x7f37a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37beffd640 1 -- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37a8004d40 msgr2=0x7f37a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37beffd640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37a8004d40 0x7f37a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.310+0000 7f37beffd640 1 -- 172.21.15.79:0/2431108886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a8003520 con 0x7f37a80039b0 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37beffd640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a80039b0 0x7f37a8003dd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f37b400c210 tx=0x7f37a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7041fb1638ed83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4007d50 con 0x7f37a80039b0 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b402e070 con 0x7f37a80039b0 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37bdffb640 1 -- 172.21.15.79:0/2431108886 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4005cd0 con 0x7f37a80039b0 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37c5815640 1 -- 172.21.15.79:0/2431108886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a80039b0 msgr2=0x7f37a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37c5815640 1 --2- 172.21.15.79:0/2431108886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37a80039b0 0x7f37a8003dd0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f37b400c210 tx=0x7f37a8005830 comp rx=0 tx=0).stop 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37bf7fe640 1 -- 172.21.15.79:0/2431108886 reap_dead start 2024-04-26T19:52:05.315 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37c5815640 1 -- 172.21.15.79:0/2431108886 shutdown_connections 2024-04-26T19:52:05.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37c5815640 1 -- 172.21.15.79:0/2431108886 >> 172.21.15.79:0/2431108886 conn(0x7f37c00b2720 msgr2=0x7f37c0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:05.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37c5815640 1 -- 172.21.15.79:0/2431108886 shutdown_connections 2024-04-26T19:52:05.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.311+0000 7f37c5815640 1 -- 172.21.15.79:0/2431108886 wait complete. 2024-04-26T19:52:05.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.312+0000 7f37c5815640 1 Processor -- start 2024-04-26T19:52:05.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37c5815640 1 -- start start 2024-04-26T19:52:05.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37c5815640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37a80039b0 0x7f37c013fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37c5815640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37c00afd50 0x7f37c00b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37c5815640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37c00b08d0 0x7f37c015e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37c5815640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f37c0145a10 con 0x7f37c00afd50 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37c5815640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37c01454f0 con 0x7f37c00b08d0 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37c5815640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f37c0145b90 con 0x7f37a80039b0 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37be7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37c00afd50 0x7f37c00b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37be7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37c00afd50 0x7f37c00b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54796/0 (socket says 172.21.15.79:54796) 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.313+0000 7f37be7fc640 1 -- 172.21.15.79:0/3786353990 learned_addr learned my addr 172.21.15.79:0/3786353990 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37beffd640 1 --2- 172.21.15.79:0/3786353990 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37a80039b0 0x7f37c013fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37bf7fe640 1 --2- 172.21.15.79:0/3786353990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37c00b08d0 0x7f37c015e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37be7fc640 1 -- 172.21.15.79:0/3786353990 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37a80039b0 msgr2=0x7f37c013fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37be7fc640 1 --2- 172.21.15.79:0/3786353990 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37a80039b0 0x7f37c013fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37be7fc640 1 -- 172.21.15.79:0/3786353990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37c00b08d0 msgr2=0x7f37c015e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37be7fc640 1 --2- 172.21.15.79:0/3786353990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37c00b08d0 0x7f37c015e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37be7fc640 1 -- 172.21.15.79:0/3786353990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c00b0d50 con 0x7f37c00afd50 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37be7fc640 1 --2- 172.21.15.79:0/3786353990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37c00afd50 0x7f37c00b0390 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f37b0011930 tx=0x7f37b0011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:05.319 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37a7fff640 1 -- 172.21.15.79:0/3786353990 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b000fa90 con 0x7f37c00afd50 2024-04-26T19:52:05.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37a7fff640 1 -- 172.21.15.79:0/3786353990 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37b000fc30 con 0x7f37c00afd50 2024-04-26T19:52:05.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37c5815640 1 -- 172.21.15.79:0/3786353990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c00aef90 con 0x7f37c00afd50 2024-04-26T19:52:05.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37a7fff640 1 -- 172.21.15.79:0/3786353990 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b0004030 con 0x7f37c00afd50 2024-04-26T19:52:05.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.314+0000 7f37c5815640 1 -- 172.21.15.79:0/3786353990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c00af550 con 0x7f37c00afd50 2024-04-26T19:52:05.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.315+0000 7f37c5815640 1 -- 172.21.15.79:0/3786353990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37c00d2590 con 0x7f37c00afd50 2024-04-26T19:52:05.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.315+0000 7f37a7fff640 1 -- 172.21.15.79:0/3786353990 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 12) ==== 90751+0+0 (secure 0 0 0) 0x7f37b0012040 con 0x7f37c00afd50 2024-04-26T19:52:05.320 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.316+0000 7f37a7fff640 1 -- 172.21.15.79:0/3786353990 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6703+0+0 (secure 0 0 0) 0x7f37b008bc90 con 0x7f37c00afd50 2024-04-26T19:52:05.321 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.320+0000 7f37a7fff640 1 -- 172.21.15.79:0/3786353990 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f37c00d2590 con 0x7f37c00afd50 2024-04-26T19:52:05.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.536+0000 7f37c5815640 1 -- 172.21.15.79:0/3786353990 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37c00506c0 con 0x7f37c00afd50 2024-04-26T19:52:05.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.537+0000 7f37a7fff640 1 -- 172.21.15.79:0/3786353990 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f37b0019070 con 0x7f37c00afd50 2024-04-26T19:52:05.538 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:05.538 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2061494256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":2061494256},{"type":"v1","addr":"172.21.15.83:6835","nonce":2061494256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/4034640420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":4034640420},{"type":"v1","addr":"172.21.15.83:6834","nonce":4034640420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/238749789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":238749789},{"type":"v1","addr":"172.21.15.79:6836","nonce":238749789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:51:43.380019+0000","modified":"2024-04-26T19:51:45.374734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/2334214524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":2334214524},{"type":"v1","addr":"172.21.15.79:6837","nonce":2334214524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5343,"qdb_cluster":[5343]},"id":2}]} 2024-04-26T19:52:05.538 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2024-04-26T19:52:05.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.539+0000 7f37c5815640 1 -- 172.21.15.79:0/3786353990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37c00afd50 msgr2=0x7f37c00b0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.539+0000 7f37c5815640 1 --2- 172.21.15.79:0/3786353990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37c00afd50 0x7f37c00b0390 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f37b0011930 tx=0x7f37b0011df0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.539+0000 7f37c5815640 1 -- 172.21.15.79:0/3786353990 shutdown_connections 2024-04-26T19:52:05.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.539+0000 7f37c5815640 1 --2- 172.21.15.79:0/3786353990 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37c00b08d0 0x7f37c015e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.539+0000 7f37c5815640 1 --2- 172.21.15.79:0/3786353990 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37c00afd50 0x7f37c00b0390 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.539+0000 7f37c5815640 1 --2- 172.21.15.79:0/3786353990 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37a80039b0 0x7f37c013fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.539+0000 7f37c5815640 1 -- 172.21.15.79:0/3786353990 >> 172.21.15.79:0/3786353990 conn(0x7f37c00b2720 msgr2=0x7f37c0140850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:05.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.539+0000 7f37c5815640 1 -- 172.21.15.79:0/3786353990 shutdown_connections 2024-04-26T19:52:05.543 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.539+0000 7f37c5815640 1 -- 172.21.15.79:0/3786353990 wait complete. 2024-04-26T19:52:05.551 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:52:05.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.647+0000 7fb01f312640 1 Processor -- start 2024-04-26T19:52:05.648 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.647+0000 7fb01f312640 1 -- start start 2024-04-26T19:52:05.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01f312640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fb018160390 0x7fb0180b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01f312640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb018160770 0x7fb0180b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01f312640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb018144ca0 con 0x7fb0181630d0 2024-04-26T19:52:05.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01f312640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fb018163e00 con 0x7fb018160770 2024-04-26T19:52:05.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01f312640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fb018144f30 con 0x7fb018160390 2024-04-26T19:52:05.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01c8ad640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb018160770 0x7fb0180b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.651 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01c8ad640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fb018160770 0x7fb0180b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54798/0 (socket says 172.21.15.79:54798) 2024-04-26T19:52:05.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01c8ad640 1 -- 172.21.15.79:0/2166088043 learned_addr learned my addr 172.21.15.79:0/2166088043 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:05.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01c8ad640 1 -- 172.21.15.79:0/2166088043 >> v2:172.21.15.83:3300/0 conn(0x7fb018160390 msgr2=0x7fb0180b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:05.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01d0ae640 1 --2- 172.21.15.79:0/2166088043 >> v2:172.21.15.83:3300/0 conn(0x7fb018160390 0x7fb0180b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01c8ad640 1 --2- 172.21.15.79:0/2166088043 >> v2:172.21.15.83:3300/0 conn(0x7fb018160390 0x7fb0180b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01c8ad640 1 -- 172.21.15.79:0/2166088043 >> v1:172.21.15.83:6789/0 conn(0x7fb0181630d0 legacy=0x7fb018163520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:05.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.648+0000 7fb01c8ad640 1 -- 172.21.15.79:0/2166088043 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb018145210 con 0x7fb018160770 2024-04-26T19:52:05.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.649+0000 7fb01c8ad640 1 --2- 172.21.15.79:0/2166088043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb018160770 0x7fb0180b4050 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb0180b5690 tx=0x7fb00802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9219c0316ab91e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:05.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.649+0000 7fb00ffff640 1 -- 172.21.15.79:0/2166088043 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008037070 con 0x7fb018160770 2024-04-26T19:52:05.652 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.649+0000 7fb00ffff640 1 -- 172.21.15.79:0/2166088043 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb008032440 con 0x7fb018160770 2024-04-26T19:52:05.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.649+0000 7fb00ffff640 1 -- 172.21.15.79:0/2166088043 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008032810 con 0x7fb018160770 2024-04-26T19:52:05.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.650+0000 7fb01f312640 1 -- 172.21.15.79:0/2166088043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb018160770 msgr2=0x7fb0180b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.650+0000 7fb01f312640 1 --2- 172.21.15.79:0/2166088043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb018160770 0x7fb0180b4050 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb0180b5690 tx=0x7fb00802a1a0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.650+0000 7fb01f312640 1 -- 172.21.15.79:0/2166088043 shutdown_connections 2024-04-26T19:52:05.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.650+0000 7fb01f312640 1 --2- 172.21.15.79:0/2166088043 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb018160770 0x7fb0180b4050 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.650+0000 7fb01f312640 1 --2- 172.21.15.79:0/2166088043 >> v2:172.21.15.83:3300/0 conn(0x7fb018160390 0x7fb0180b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.650+0000 7fb01f312640 1 -- 172.21.15.79:0/2166088043 >> 172.21.15.79:0/2166088043 conn(0x7fb0180b2720 msgr2=0x7fb018070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:05.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.650+0000 7fb01f312640 1 -- 172.21.15.79:0/2166088043 shutdown_connections 2024-04-26T19:52:05.653 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.650+0000 7fb01f312640 1 -- 172.21.15.79:0/2166088043 wait complete. 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.651+0000 7fb01f312640 1 Processor -- start 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.651+0000 7fb01f312640 1 -- start start 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01f312640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb018160390 0x7fb0180b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01f312640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb018160770 0x7fb018145da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01f312640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0181630d0 0x7fb01812e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01f312640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fb018145b90 con 0x7fb0181630d0 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01f312640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fb018145a10 con 0x7fb018160770 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01f312640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fb018169460 con 0x7fb018160390 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d8af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0181630d0 0x7fb01812e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.658 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d8af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0181630d0 0x7fb01812e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54804/0 (socket says 172.21.15.79:54804) 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d8af640 1 -- 172.21.15.79:0/643695679 learned_addr learned my addr 172.21.15.79:0/643695679 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d0ae640 1 --2- 172.21.15.79:0/643695679 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb018160390 0x7fb0180b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01c8ad640 1 --2- 172.21.15.79:0/643695679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb018160770 0x7fb018145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d8af640 1 -- 172.21.15.79:0/643695679 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb018160390 msgr2=0x7fb0180b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d8af640 1 --2- 172.21.15.79:0/643695679 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb018160390 0x7fb0180b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d8af640 1 -- 172.21.15.79:0/643695679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb018160770 msgr2=0x7fb018145da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d8af640 1 --2- 172.21.15.79:0/643695679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb018160770 0x7fb018145da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d8af640 1 -- 172.21.15.79:0/643695679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0181412e0 con 0x7fb0181630d0 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01d8af640 1 --2- 172.21.15.79:0/643695679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0181630d0 0x7fb01812e850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c3c0 tx=0x7fb01000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb00dffb640 1 -- 172.21.15.79:0/643695679 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb010017070 con 0x7fb0181630d0 2024-04-26T19:52:05.659 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb00dffb640 1 -- 172.21.15.79:0/643695679 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0100035f0 con 0x7fb0181630d0 2024-04-26T19:52:05.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb01f312640 1 -- 172.21.15.79:0/643695679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb018141580 con 0x7fb0181630d0 2024-04-26T19:52:05.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.652+0000 7fb00dffb640 1 -- 172.21.15.79:0/643695679 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0100039e0 con 0x7fb0181630d0 2024-04-26T19:52:05.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.653+0000 7fb01f312640 1 -- 172.21.15.79:0/643695679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb018141b40 con 0x7fb0181630d0 2024-04-26T19:52:05.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.654+0000 7fb01f312640 1 -- 172.21.15.79:0/643695679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0180506c0 con 0x7fb0181630d0 2024-04-26T19:52:05.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.654+0000 7fb00dffb640 1 -- 172.21.15.79:0/643695679 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 12) ==== 90751+0+0 (secure 0 0 0) 0x7fb01001a070 con 0x7fb0181630d0 2024-04-26T19:52:05.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.654+0000 7fb00dffb640 1 -- 172.21.15.79:0/643695679 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6703+0+0 (secure 0 0 0) 0x7fb01008acc0 con 0x7fb0181630d0 2024-04-26T19:52:05.660 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.659+0000 7fb00dffb640 1 -- 172.21.15.79:0/643695679 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fb010058b10 con 0x7fb0181630d0 2024-04-26T19:52:05.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.834+0000 7fb01f312640 1 -- 172.21.15.79:0/643695679 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb01806cd50 con 0x7fb0181630d0 2024-04-26T19:52:05.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.835+0000 7fb00dffb640 1 -- 172.21.15.79:0/643695679 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17770 (secure 0 0 0) 0x7fb01005ce10 con 0x7fb0181630d0 2024-04-26T19:52:05.836 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:05.836 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":41,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:52:05.198412+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:51:41.583741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T19:51:42.339687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:0/2026733823":"2024-04-27T19:52:05.198362+0000","172.21.15.79:6832/3161939478":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1948040587":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/3742475428":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1992052263":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/3466651560":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/1084811462":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/16732380":"2024-04-27T19:51:58.575815+0000","172.21.15.79:6832/3922824612":"2024-04-27T19:52:05.198362+0000","172.21.15.79:6833/3161939478":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1790989087":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/2471022702":"2024-04-27T19:52:05.198362+0000","172.21.15.79:6833/3922824612":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/466097619":"2024-04-27T19:52:05.198362+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:52:05.837 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.837+0000 7fb01f312640 1 -- 172.21.15.79:0/643695679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0181630d0 msgr2=0x7fb01812e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.837+0000 7fb01f312640 1 --2- 172.21.15.79:0/643695679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0181630d0 0x7fb01812e850 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb01000c3c0 tx=0x7fb01000c880 comp rx=0 tx=0).stop 2024-04-26T19:52:05.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.837+0000 7fb01f312640 1 -- 172.21.15.79:0/643695679 shutdown_connections 2024-04-26T19:52:05.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.837+0000 7fb01f312640 1 --2- 172.21.15.79:0/643695679 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fb0181630d0 0x7fb01812e850 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.837+0000 7fb01f312640 1 --2- 172.21.15.79:0/643695679 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fb018160770 0x7fb018145da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.837+0000 7fb01f312640 1 --2- 172.21.15.79:0/643695679 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fb018160390 0x7fb0180b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.837+0000 7fb01f312640 1 -- 172.21.15.79:0/643695679 >> 172.21.15.79:0/643695679 conn(0x7fb0180b2720 msgr2=0x7fb018143a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:05.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.837+0000 7fb01f312640 1 -- 172.21.15.79:0/643695679 shutdown_connections 2024-04-26T19:52:05.838 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.837+0000 7fb01f312640 1 -- 172.21.15.79:0/643695679 wait complete. 2024-04-26T19:52:05.848 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T19:52:05.849 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:52:05.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.946+0000 7fd307e52640 1 Processor -- start 2024-04-26T19:52:05.947 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.946+0000 7fd307e52640 1 -- start start 2024-04-26T19:52:05.948 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd307e52640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd30006ff00 0x7fd300148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd307e52640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3000d2cd0 0x7fd3000d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd307e52640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3000b1290 con 0x7fd30006fb20 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd307e52640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd3000d3140 con 0x7fd30006ff00 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd307e52640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd3000d3e60 con 0x7fd3000d2cd0 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd3053ed640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd30006ff00 0x7fd300148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd3053ed640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd30006ff00 0x7fd300148fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54810/0 (socket says 172.21.15.79:54810) 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd3053ed640 1 -- 172.21.15.79:0/4162253924 learned_addr learned my addr 172.21.15.79:0/4162253924 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd3063ef640 1 --2- 172.21.15.79:0/4162253924 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3000d2cd0 0x7fd3000d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd3053ed640 1 -- 172.21.15.79:0/4162253924 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3000d2cd0 msgr2=0x7fd3000d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd3053ed640 1 --2- 172.21.15.79:0/4162253924 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3000d2cd0 0x7fd3000d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd3053ed640 1 -- 172.21.15.79:0/4162253924 >> v1:172.21.15.83:6789/0 conn(0x7fd30006fb20 legacy=0x7fd3001488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.947+0000 7fd3053ed640 1 -- 172.21.15.79:0/4162253924 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3000d4090 con 0x7fd30006ff00 2024-04-26T19:52:05.949 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.948+0000 7fd3053ed640 1 --2- 172.21.15.79:0/4162253924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd30006ff00 0x7fd300148fd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009870 tx=0x7fd2f002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728c44b163b46d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:05.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.948+0000 7fd304bec640 1 -- 172.21.15.79:0/4162253924 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0037070 con 0x7fd30006ff00 2024-04-26T19:52:05.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.948+0000 7fd304bec640 1 -- 172.21.15.79:0/4162253924 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2f002ae50 con 0x7fd30006ff00 2024-04-26T19:52:05.950 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.948+0000 7fd304bec640 1 -- 172.21.15.79:0/4162253924 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0032860 con 0x7fd30006ff00 2024-04-26T19:52:05.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.949+0000 7fd307e52640 1 -- 172.21.15.79:0/4162253924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd30006ff00 msgr2=0x7fd300148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.949+0000 7fd307e52640 1 --2- 172.21.15.79:0/4162253924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd30006ff00 0x7fd300148fd0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009870 tx=0x7fd2f002a300 comp rx=0 tx=0).stop 2024-04-26T19:52:05.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.949+0000 7fd307e52640 1 -- 172.21.15.79:0/4162253924 shutdown_connections 2024-04-26T19:52:05.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.949+0000 7fd307e52640 1 --2- 172.21.15.79:0/4162253924 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd3000d2cd0 0x7fd3000d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.949+0000 7fd307e52640 1 --2- 172.21.15.79:0/4162253924 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd30006ff00 0x7fd300148fd0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.949+0000 7fd307e52640 1 -- 172.21.15.79:0/4162253924 >> 172.21.15.79:0/4162253924 conn(0x7fd300071c30 msgr2=0x7fd30006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:05.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.949+0000 7fd307e52640 1 -- 172.21.15.79:0/4162253924 shutdown_connections 2024-04-26T19:52:05.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.949+0000 7fd307e52640 1 -- 172.21.15.79:0/4162253924 wait complete. 2024-04-26T19:52:05.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.950+0000 7fd307e52640 1 Processor -- start 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.950+0000 7fd307e52640 1 -- start start 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd307e52640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd30006fb20 0x7fd300142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd307e52640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd30006ff00 0x7fd300138a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd307e52640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3000d2cd0 0x7fd3001f9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd307e52640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd3000d4930 con 0x7fd3000d2cd0 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd307e52640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd3000d4650 con 0x7fd30006fb20 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd307e52640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd3000d4c10 con 0x7fd30006ff00 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3063ef640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3000d2cd0 0x7fd3001f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3063ef640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3000d2cd0 0x7fd3001f9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54826/0 (socket says 172.21.15.79:54826) 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3063ef640 1 -- 172.21.15.79:0/2173949434 learned_addr learned my addr 172.21.15.79:0/2173949434 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3053ed640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd30006ff00 0x7fd300138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd305bee640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd30006fb20 0x7fd300142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3063ef640 1 -- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd30006ff00 msgr2=0x7fd300138a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3063ef640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd30006ff00 0x7fd300138a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3063ef640 1 -- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd30006fb20 msgr2=0x7fd300142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3063ef640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd30006fb20 0x7fd300142110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3063ef640 1 -- 172.21.15.79:0/2173949434 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3001f9b70 con 0x7fd3000d2cd0 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3053ed640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd30006ff00 0x7fd300138a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd305bee640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd30006fb20 0x7fd300142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd3063ef640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3000d2cd0 0x7fd3001f9630 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009d30 tx=0x7fd2f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd2f67fc640 1 -- 172.21.15.79:0/2173949434 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8018020 con 0x7fd3000d2cd0 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd2f67fc640 1 -- 172.21.15.79:0/2173949434 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2f8014480 con 0x7fd3000d2cd0 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd307e52640 1 -- 172.21.15.79:0/2173949434 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30015b260 con 0x7fd3000d2cd0 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd2f67fc640 1 -- 172.21.15.79:0/2173949434 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8014870 con 0x7fd3000d2cd0 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.951+0000 7fd307e52640 1 -- 172.21.15.79:0/2173949434 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd30015b820 con 0x7fd3000d2cd0 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.952+0000 7fd307e52640 1 -- 172.21.15.79:0/2173949434 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3000506c0 con 0x7fd3000d2cd0 2024-04-26T19:52:05.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.953+0000 7fd2f67fc640 1 -- 172.21.15.79:0/2173949434 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 12) ==== 90751+0+0 (secure 0 0 0) 0x7fd2f8005d10 con 0x7fd3000d2cd0 2024-04-26T19:52:05.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.953+0000 7fd2f67fc640 1 -- 172.21.15.79:0/2173949434 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6703+0+0 (secure 0 0 0) 0x7fd2f808c3e0 con 0x7fd3000d2cd0 2024-04-26T19:52:05.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:05.957+0000 7fd2f67fc640 1 -- 172.21.15.79:0/2173949434 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd2f8022070 con 0x7fd3000d2cd0 2024-04-26T19:52:06.169 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.169+0000 7fd307e52640 1 -- 172.21.15.79:0/2173949434 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3000d3700 con 0x7fd3000d2cd0 2024-04-26T19:52:06.170 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.169+0000 7fd2f67fc640 1 -- 172.21.15.79:0/2173949434 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7fd3001f9b70 con 0x7fd3000d2cd0 2024-04-26T19:52:06.170 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:06.170 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2061494256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":2061494256},{"type":"v1","addr":"172.21.15.83:6835","nonce":2061494256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/4034640420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":4034640420},{"type":"v1","addr":"172.21.15.83:6834","nonce":4034640420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/238749789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":238749789},{"type":"v1","addr":"172.21.15.79:6836","nonce":238749789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:51:43.380019+0000","modified":"2024-04-26T19:51:45.374734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/2334214524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":2334214524},{"type":"v1","addr":"172.21.15.79:6837","nonce":2334214524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5343,"qdb_cluster":[5343]},"id":2}]} 2024-04-26T19:52:06.170 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2024-04-26T19:52:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.171+0000 7fd307e52640 1 -- 172.21.15.79:0/2173949434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3000d2cd0 msgr2=0x7fd3001f9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.171+0000 7fd307e52640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3000d2cd0 0x7fd3001f9630 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009d30 tx=0x7fd2f800c490 comp rx=0 tx=0).stop 2024-04-26T19:52:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.171+0000 7fd307e52640 1 -- 172.21.15.79:0/2173949434 shutdown_connections 2024-04-26T19:52:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.171+0000 7fd307e52640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd3000d2cd0 0x7fd3001f9630 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.171+0000 7fd307e52640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd30006ff00 0x7fd300138a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.171+0000 7fd307e52640 1 --2- 172.21.15.79:0/2173949434 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd30006fb20 0x7fd300142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.171+0000 7fd307e52640 1 -- 172.21.15.79:0/2173949434 >> 172.21.15.79:0/2173949434 conn(0x7fd300071c30 msgr2=0x7fd300148a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.171+0000 7fd307e52640 1 -- 172.21.15.79:0/2173949434 shutdown_connections 2024-04-26T19:52:06.172 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.171+0000 7fd307e52640 1 -- 172.21.15.79:0/2173949434 wait complete. 2024-04-26T19:52:06.183 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:52:06.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.283+0000 7f060aa64640 1 Processor -- start 2024-04-26T19:52:06.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.283+0000 7f060aa64640 1 -- start start 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f060aa64640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f060406ff00 0x7f0604148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f060aa64640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f06040d2cd0 0x7f06040d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f060aa64640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06040b1290 con 0x7f060406fb20 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f060aa64640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f06040d3140 con 0x7f060406ff00 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f060aa64640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f06040d3e60 con 0x7f06040d2cd0 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f0603fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f060406ff00 0x7f0604148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f0603fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f060406ff00 0x7f0604148fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54828/0 (socket says 172.21.15.79:54828) 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f0603fff640 1 -- 172.21.15.79:0/581579264 learned_addr learned my addr 172.21.15.79:0/581579264 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f0609001640 1 --2- 172.21.15.79:0/581579264 >> v2:172.21.15.83:3300/0 conn(0x7f06040d2cd0 0x7f06040d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f0603fff640 1 -- 172.21.15.79:0/581579264 >> v2:172.21.15.83:3300/0 conn(0x7f06040d2cd0 msgr2=0x7f06040d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f0603fff640 1 --2- 172.21.15.79:0/581579264 >> v2:172.21.15.83:3300/0 conn(0x7f06040d2cd0 0x7f06040d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f0603fff640 1 -- 172.21.15.79:0/581579264 >> v1:172.21.15.83:6789/0 conn(0x7f060406fb20 legacy=0x7f06041488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.284+0000 7f0603fff640 1 -- 172.21.15.79:0/581579264 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06040d4090 con 0x7f060406ff00 2024-04-26T19:52:06.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.285+0000 7f0603fff640 1 --2- 172.21.15.79:0/581579264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 0x7f0604148fd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f06040b0af0 tx=0x7f05f002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe704586000b81dc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:06.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.285+0000 7f06037fe640 1 -- 172.21.15.79:0/581579264 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0037070 con 0x7f060406ff00 2024-04-26T19:52:06.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.285+0000 7f06037fe640 1 -- 172.21.15.79:0/581579264 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f002ae50 con 0x7f060406ff00 2024-04-26T19:52:06.287 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.285+0000 7f06037fe640 1 -- 172.21.15.79:0/581579264 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0032860 con 0x7f060406ff00 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.286+0000 7f060aa64640 1 -- 172.21.15.79:0/581579264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 msgr2=0x7f0604148fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.286+0000 7f060aa64640 1 --2- 172.21.15.79:0/581579264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 0x7f0604148fd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f06040b0af0 tx=0x7f05f002a300 comp rx=0 tx=0).stop 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.286+0000 7f060aa64640 1 -- 172.21.15.79:0/581579264 shutdown_connections 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.286+0000 7f060aa64640 1 --2- 172.21.15.79:0/581579264 >> v2:172.21.15.83:3300/0 conn(0x7f06040d2cd0 0x7f06040d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.286+0000 7f060aa64640 1 --2- 172.21.15.79:0/581579264 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 0x7f0604148fd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.286+0000 7f060aa64640 1 -- 172.21.15.79:0/581579264 >> 172.21.15.79:0/581579264 conn(0x7f0604071c30 msgr2=0x7f060406dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.286+0000 7f060aa64640 1 -- 172.21.15.79:0/581579264 shutdown_connections 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.286+0000 7f060aa64640 1 -- 172.21.15.79:0/581579264 wait complete. 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.287+0000 7f060aa64640 1 Processor -- start 2024-04-26T19:52:06.288 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.287+0000 7f060aa64640 1 -- start start 2024-04-26T19:52:06.290 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f060aa64640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060406fb20 0x7f06040c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f060aa64640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 0x7f0604141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f060aa64640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06040d2cd0 0x7f06041f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f060aa64640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f06040d4930 con 0x7f060406ff00 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f060aa64640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f06040d4650 con 0x7f060406fb20 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f060aa64640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f06040d4c10 con 0x7f06040d2cd0 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0603fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 0x7f0604141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0603fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 0x7f0604141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54834/0 (socket says 172.21.15.79:54834) 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0603fff640 1 -- 172.21.15.79:0/828304646 learned_addr learned my addr 172.21.15.79:0/828304646 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0608800640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060406fb20 0x7f06040c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0609001640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06040d2cd0 0x7f06041f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0603fff640 1 -- 172.21.15.79:0/828304646 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06040d2cd0 msgr2=0x7f06041f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0603fff640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06040d2cd0 0x7f06041f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.291 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0603fff640 1 -- 172.21.15.79:0/828304646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060406fb20 msgr2=0x7f06040c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0603fff640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060406fb20 0x7f06040c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.288+0000 7f0603fff640 1 -- 172.21.15.79:0/828304646 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0604142470 con 0x7f060406ff00 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.289+0000 7f0603fff640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 0x7f0604141f30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f05f002a340 tx=0x7f05f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.289+0000 7f06017fa640 1 -- 172.21.15.79:0/828304646 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0041020 con 0x7f060406ff00 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.289+0000 7f06017fa640 1 -- 172.21.15.79:0/828304646 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05f003c870 con 0x7f060406ff00 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.289+0000 7f06017fa640 1 -- 172.21.15.79:0/828304646 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0037040 con 0x7f060406ff00 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.289+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0604161270 con 0x7f060406ff00 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.289+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0604161830 con 0x7f060406ff00 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.290+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06040506c0 con 0x7f060406ff00 2024-04-26T19:52:06.292 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.290+0000 7f06017fa640 1 -- 172.21.15.79:0/828304646 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 90878+0+0 (secure 0 0 0) 0x7f05f0044020 con 0x7f060406ff00 2024-04-26T19:52:06.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.291+0000 7f06017fa640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f05e806c740 0x7f05e806ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.291+0000 7f06017fa640 1 -- 172.21.15.79:0/828304646 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6703+0+0 (secure 0 0 0) 0x7f05f00ac4e0 con 0x7f060406ff00 2024-04-26T19:52:06.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.291+0000 7f0608800640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f05e806c740 0x7f05e806ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.293 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.291+0000 7f0608800640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f05e806c740 0x7f05e806ec00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f05f8001640 tx=0x7f05f8009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:06.295 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.295+0000 7f06017fa640 1 -- 172.21.15.79:0/828304646 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f05f004b030 con 0x7f060406ff00 2024-04-26T19:52:06.505 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.505+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f06040d3700 con 0x7f060406ff00 2024-04-26T19:52:06.506 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.505+0000 7f06017fa640 1 -- 172.21.15.79:0/828304646 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5178 (secure 0 0 0) 0x7f05f003cc60 con 0x7f060406ff00 2024-04-26T19:52:06.506 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:06.506 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2061494256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":2061494256},{"type":"v1","addr":"172.21.15.83:6835","nonce":2061494256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/4034640420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":4034640420},{"type":"v1","addr":"172.21.15.83:6834","nonce":4034640420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/238749789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":238749789},{"type":"v1","addr":"172.21.15.79:6836","nonce":238749789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T19:51:43.380019+0000","modified":"2024-04-26T19:51:45.374734+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5343},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5343":{"gid":5343,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.79:6837/2334214524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":2334214524},{"type":"v1","addr":"172.21.15.79:6837","nonce":2334214524}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5343,"qdb_cluster":[5343]},"id":2}]} 2024-04-26T19:52:06.506 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 16 2024-04-26T19:52:06.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f05e806c740 msgr2=0x7f05e806ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f05e806c740 0x7f05e806ec00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f05f8001640 tx=0x7f05f8009040 comp rx=0 tx=0).stop 2024-04-26T19:52:06.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 msgr2=0x7f0604141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.508 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 0x7f0604141f30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f05f002a340 tx=0x7f05f00046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 shutdown_connections 2024-04-26T19:52:06.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f05e806c740 0x7f05e806ec00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f06040d2cd0 0x7f06041f52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f060406ff00 0x7f0604141f30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 --2- 172.21.15.79:0/828304646 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f060406fb20 0x7f06040c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 >> 172.21.15.79:0/828304646 conn(0x7f0604071c30 msgr2=0x7f060406e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:06.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 shutdown_connections 2024-04-26T19:52:06.509 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.507+0000 7f060aa64640 1 -- 172.21.15.79:0/828304646 wait complete. 2024-04-26T19:52:06.519 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:52:06.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.616+0000 7feed4c74640 1 Processor -- start 2024-04-26T19:52:06.625 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.616+0000 7feed4c74640 1 -- start start 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feed4c74640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7feed006fb20 0x7feed00d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feed4c74640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed00b1290 con 0x7feed0148fd0 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feed4c74640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed00d4090 con 0x7feed006ff00 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feed4c74640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7feed00d4370 con 0x7feed006fb20 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feeced9d640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7feed0148fd0 0x7feed00aebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59784/0 (socket says 172.21.15.79:59784) 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feeced9d640 1 -- 172.21.15.79:0/4106215680 learned_addr learned my addr 172.21.15.79:0/4106215680 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 12417020 0 0) 0x7feed00b1290 con 0x7feed0148fd0 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feebc003520 con 0x7feed0148fd0 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feece59c640 1 --2- 172.21.15.79:0/4106215680 >> v2:172.21.15.83:3300/0 conn(0x7feed006fb20 0x7feed00d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.626 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2947738 0 0) 0x7feebc003520 con 0x7feed0148fd0 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 >> v2:172.21.15.83:3300/0 conn(0x7feed006fb20 msgr2=0x7feed00d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feecdd9b640 1 --2- 172.21.15.79:0/4106215680 >> v2:172.21.15.83:3300/0 conn(0x7feed006fb20 0x7feed00d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 >> v1:172.21.15.83:6789/0 conn(0x7feed006ff00 legacy=0x7feed01488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed00d4650 con 0x7feed0148fd0 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.617+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7feec80019f0 con 0x7feed0148fd0 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 >> v1:172.21.15.79:6789/0 conn(0x7feed0148fd0 legacy=0x7feed00aebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feecdd9b640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feebc0039e0 0x7feebc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feecdd9b640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feebc004800 0x7feebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feecdd9b640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc0051c0 0x7feebc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feed00d4370 con 0x7feebc004800 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feebc00c070 con 0x7feebc0051c0 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7feebc00c270 con 0x7feebc0039e0 2024-04-26T19:52:06.627 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feec7fff640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feebc004800 0x7feebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feece59c640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feebc0039e0 0x7feebc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feeced9d640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc0051c0 0x7feebc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feec7fff640 1 -- 172.21.15.79:0/4106215680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feebc0039e0 msgr2=0x7feebc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feec7fff640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feebc0039e0 0x7feebc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feec7fff640 1 -- 172.21.15.79:0/4106215680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc0051c0 msgr2=0x7feebc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feec7fff640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc0051c0 0x7feebc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feeced9d640 1 -- 172.21.15.79:0/4106215680 reap_dead start 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.618+0000 7feec7fff640 1 -- 172.21.15.79:0/4106215680 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feebc003520 con 0x7feebc004800 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feec7fff640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feebc004800 0x7feebc004c80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7feeb8001750 tx=0x7feebc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e222df1636407874 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb8001370 con 0x7feebc004800 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb80052d0 con 0x7feebc004800 2024-04-26T19:52:06.628 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feecdd9b640 1 -- 172.21.15.79:0/4106215680 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb802abc0 con 0x7feebc004800 2024-04-26T19:52:06.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feed4c74640 1 -- 172.21.15.79:0/4106215680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feebc004800 msgr2=0x7feebc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feed4c74640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feebc004800 0x7feebc004c80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7feeb8001750 tx=0x7feebc005830 comp rx=0 tx=0).stop 2024-04-26T19:52:06.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feed4c74640 1 -- 172.21.15.79:0/4106215680 shutdown_connections 2024-04-26T19:52:06.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feed4c74640 1 --2- 172.21.15.79:0/4106215680 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feebc004800 0x7feebc004c80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feed4c74640 1 -- 172.21.15.79:0/4106215680 >> 172.21.15.79:0/4106215680 conn(0x7feed0071c30 msgr2=0x7feed006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:06.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feed4c74640 1 -- 172.21.15.79:0/4106215680 shutdown_connections 2024-04-26T19:52:06.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.619+0000 7feed4c74640 1 -- 172.21.15.79:0/4106215680 wait complete. 2024-04-26T19:52:06.629 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.621+0000 7feed4c74640 1 Processor -- start 2024-04-26T19:52:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.621+0000 7feed4c74640 1 -- start start 2024-04-26T19:52:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.621+0000 7feed4c74640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc004800 0x7feed013f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.621+0000 7feed4c74640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feed0140ec0 0x7feed01462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.621+0000 7feed4c74640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feed015ede0 0x7feed0146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.621+0000 7feed4c74640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7feed00d2cd0 con 0x7feed015ede0 2024-04-26T19:52:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.621+0000 7feed4c74640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7feed00d4c10 con 0x7feebc004800 2024-04-26T19:52:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.621+0000 7feed4c74640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7feed00d2e50 con 0x7feed0140ec0 2024-04-26T19:52:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.621+0000 7feeced9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feed015ede0 0x7feed0146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.630 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feeced9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feed015ede0 0x7feed0146810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54854/0 (socket says 172.21.15.79:54854) 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feeced9d640 1 -- 172.21.15.79:0/3338515929 learned_addr learned my addr 172.21.15.79:0/3338515929 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feec7fff640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feed0140ec0 0x7feed01462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feece59c640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc004800 0x7feed013f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feeced9d640 1 -- 172.21.15.79:0/3338515929 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feed0140ec0 msgr2=0x7feed01462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feeced9d640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feed0140ec0 0x7feed01462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feeced9d640 1 -- 172.21.15.79:0/3338515929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc004800 msgr2=0x7feed013f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feeced9d640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc004800 0x7feed013f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feeced9d640 1 -- 172.21.15.79:0/3338515929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed013fe20 con 0x7feed015ede0 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feeced9d640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feed015ede0 0x7feed0146810 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feec8000f80 tx=0x7feec8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feec6ffd640 1 -- 172.21.15.79:0/3338515929 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec800aa00 con 0x7feed015ede0 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feec6ffd640 1 -- 172.21.15.79:0/3338515929 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8019070 con 0x7feed015ede0 2024-04-26T19:52:06.631 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed0140100 con 0x7feed015ede0 2024-04-26T19:52:06.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feec6ffd640 1 -- 172.21.15.79:0/3338515929 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec800ade0 con 0x7feed015ede0 2024-04-26T19:52:06.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.622+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed01406c0 con 0x7feed015ede0 2024-04-26T19:52:06.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.623+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed00506c0 con 0x7feed015ede0 2024-04-26T19:52:06.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.624+0000 7feec6ffd640 1 -- 172.21.15.79:0/3338515929 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 90878+0+0 (secure 0 0 0) 0x7feec801e460 con 0x7feed015ede0 2024-04-26T19:52:06.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.624+0000 7feec6ffd640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feeb006c740 0x7feeb006ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.624+0000 7feec6ffd640 1 -- 172.21.15.79:0/3338515929 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6703+0+0 (secure 0 0 0) 0x7feec808be50 con 0x7feed015ede0 2024-04-26T19:52:06.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.624+0000 7feece59c640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feeb006c740 0x7feeb006ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.624+0000 7feece59c640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feeb006c740 0x7feeb006ec00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feec0009c40 tx=0x7feec0009190 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:06.632 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.628+0000 7feec6ffd640 1 -- 172.21.15.79:0/3338515929 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feec805a1e0 con 0x7feed015ede0 2024-04-26T19:52:06.805 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.804+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feed00d2ab0 con 0x7feed015ede0 2024-04-26T19:52:06.806 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.806+0000 7feec6ffd640 1 -- 172.21.15.79:0/3338515929 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17770 (secure 0 0 0) 0x7feec8059ae0 con 0x7feed015ede0 2024-04-26T19:52:06.806 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:06.807 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":41,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:52:05.198412+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-26T19:51:41.583741+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-26T19:51:42.339687+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:0/2026733823":"2024-04-27T19:52:05.198362+0000","172.21.15.79:6832/3161939478":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1948040587":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/3742475428":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1992052263":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/3466651560":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/1084811462":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/16732380":"2024-04-27T19:51:58.575815+0000","172.21.15.79:6832/3922824612":"2024-04-27T19:52:05.198362+0000","172.21.15.79:6833/3161939478":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1790989087":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/2471022702":"2024-04-27T19:52:05.198362+0000","172.21.15.79:6833/3922824612":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/466097619":"2024-04-27T19:52:05.198362+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:52:06.808 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.807+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feeb006c740 msgr2=0x7feeb006ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.807+0000 7feed4c74640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feeb006c740 0x7feeb006ec00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feec0009c40 tx=0x7feec0009190 comp rx=0 tx=0).stop 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.807+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feed015ede0 msgr2=0x7feed0146810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.807+0000 7feed4c74640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feed015ede0 0x7feed0146810 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7feec8000f80 tx=0x7feec8007a40 comp rx=0 tx=0).stop 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.808+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 shutdown_connections 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.808+0000 7feed4c74640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feeb006c740 0x7feeb006ec00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.808+0000 7feed4c74640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7feed015ede0 0x7feed0146810 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.808+0000 7feed4c74640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7feed0140ec0 0x7feed01462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.808+0000 7feed4c74640 1 --2- 172.21.15.79:0/3338515929 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7feebc004800 0x7feed013f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.808+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 >> 172.21.15.79:0/3338515929 conn(0x7feed0071c30 msgr2=0x7feed00b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.808+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 shutdown_connections 2024-04-26T19:52:06.809 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.808+0000 7feed4c74640 1 -- 172.21.15.79:0/3338515929 wait complete. 2024-04-26T19:52:06.820 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-26T19:52:06.916 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.916+0000 7f2ddcdbf640 1 Processor -- start 2024-04-26T19:52:06.917 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.916+0000 7f2ddcdbf640 1 -- start start 2024-04-26T19:52:06.919 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.916+0000 7f2ddcdbf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd80ced80 0x7f2dd8071810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.916+0000 7f2ddcdbf640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2dd8071d50 0x7f2dd815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2ddcdbf640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dd80cf310 con 0x7f2dd815b850 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2ddcdbf640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f2dd80cf5f0 con 0x7f2dd8071d50 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2ddcdbf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd80cf8d0 con 0x7f2dd80ced80 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd6d9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2dd8071d50 0x7f2dd815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd659c640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f2dd815b850 0x7f2dd80d03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:59796/0 (socket says 172.21.15.79:59796) 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd6d9d640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2dd8071d50 0x7f2dd815f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54868/0 (socket says 172.21.15.79:54868) 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd659c640 1 -- 172.21.15.79:0/3435187943 learned_addr learned my addr 172.21.15.79:0/3435187943 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd5d9b640 1 --2- 172.21.15.79:0/3435187943 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd80ced80 0x7f2dd8071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd6d9d640 1 -- 172.21.15.79:0/3435187943 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd80ced80 msgr2=0x7f2dd8071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd6d9d640 1 --2- 172.21.15.79:0/3435187943 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd80ced80 0x7f2dd8071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd6d9d640 1 -- 172.21.15.79:0/3435187943 >> v1:172.21.15.79:6789/0 conn(0x7f2dd815b850 legacy=0x7f2dd80d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.920 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd6d9d640 1 -- 172.21.15.79:0/3435187943 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd80cfbb0 con 0x7f2dd8071d50 2024-04-26T19:52:06.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.917+0000 7f2dd559a640 1 -- 172.21.15.79:0/3435187943 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1955407833 0 0) 0x7f2dd80cf310 con 0x7f2dd815b850 2024-04-26T19:52:06.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2dd6d9d640 1 --2- 172.21.15.79:0/3435187943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd8071d50 0x7f2dd815f4e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009d90 tx=0x7f2dd002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cff6482f260a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:06.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2dd559a640 1 -- 172.21.15.79:0/3435187943 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0031600 con 0x7f2dd8071d50 2024-04-26T19:52:06.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2dd559a640 1 -- 172.21.15.79:0/3435187943 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd00317a0 con 0x7f2dd8071d50 2024-04-26T19:52:06.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2dd559a640 1 -- 172.21.15.79:0/3435187943 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0031b70 con 0x7f2dd8071d50 2024-04-26T19:52:06.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/3435187943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd8071d50 msgr2=0x7f2dd815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2ddcdbf640 1 --2- 172.21.15.79:0/3435187943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd8071d50 0x7f2dd815f4e0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009d90 tx=0x7f2dd002a5d0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.921 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/3435187943 shutdown_connections 2024-04-26T19:52:06.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2ddcdbf640 1 --2- 172.21.15.79:0/3435187943 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd8071d50 0x7f2dd815f4e0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2ddcdbf640 1 --2- 172.21.15.79:0/3435187943 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd80ced80 0x7f2dd8071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/3435187943 >> 172.21.15.79:0/3435187943 conn(0x7f2dd80b39d0 msgr2=0x7f2dd806feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:06.922 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/3435187943 shutdown_connections 2024-04-26T19:52:06.923 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.918+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/3435187943 wait complete. 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.920+0000 7f2ddcdbf640 1 Processor -- start 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.920+0000 7f2ddcdbf640 1 -- start start 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.920+0000 7f2ddcdbf640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd8071d50 0x7f2dd806dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.920+0000 7f2ddcdbf640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd80ced80 0x7f2dd81f0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.920+0000 7f2ddcdbf640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd815b850 0x7f2dd8158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.920+0000 7f2ddcdbf640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd815bc30 con 0x7f2dd80ced80 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.920+0000 7f2ddcdbf640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd80cd9d0 con 0x7f2dd815b850 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.920+0000 7f2ddcdbf640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd815bdb0 con 0x7f2dd8071d50 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd5d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd80ced80 0x7f2dd81f0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd5d9b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd80ced80 0x7f2dd81f0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54870/0 (socket says 172.21.15.79:54870) 2024-04-26T19:52:06.926 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd5d9b640 1 -- 172.21.15.79:0/390350644 learned_addr learned my addr 172.21.15.79:0/390350644 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd659c640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd8071d50 0x7f2dd806dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd6d9d640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd815b850 0x7f2dd8158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd5d9b640 1 -- 172.21.15.79:0/390350644 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd8071d50 msgr2=0x7f2dd806dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd5d9b640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd8071d50 0x7f2dd806dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd5d9b640 1 -- 172.21.15.79:0/390350644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd815b850 msgr2=0x7f2dd8158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd5d9b640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd815b850 0x7f2dd8158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd5d9b640 1 -- 172.21.15.79:0/390350644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd8159f80 con 0x7f2dd80ced80 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dd5d9b640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd80ced80 0x7f2dd81f0870 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800e9b0 tx=0x7f2dc800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dceffd640 1 -- 172.21.15.79:0/390350644 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8017070 con 0x7f2dd80ced80 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dceffd640 1 -- 172.21.15.79:0/390350644 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc800c9a0 con 0x7f2dd80ced80 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd815a260 con 0x7f2dd80ced80 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2dceffd640 1 -- 172.21.15.79:0/390350644 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc800cd70 con 0x7f2dd80ced80 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.921+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd8159380 con 0x7f2dd80ced80 2024-04-26T19:52:06.927 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.922+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd80506c0 con 0x7f2dd80ced80 2024-04-26T19:52:06.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.923+0000 7f2dceffd640 1 -- 172.21.15.79:0/390350644 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 90878+0+0 (secure 0 0 0) 0x7f2dc801a070 con 0x7f2dd80ced80 2024-04-26T19:52:06.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.923+0000 7f2dceffd640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2db806c740 0x7f2db806ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:06.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.923+0000 7f2dceffd640 1 -- 172.21.15.79:0/390350644 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6703+0+0 (secure 0 0 0) 0x7f2dc808be60 con 0x7f2dd80ced80 2024-04-26T19:52:06.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.923+0000 7f2dd659c640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2db806c740 0x7f2db806ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:06.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.924+0000 7f2dd659c640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2db806c740 0x7f2db806ec00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0007fd0 tx=0x7f2dc0007de0 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:06.928 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:06.928+0000 7f2dceffd640 1 -- 172.21.15.79:0/390350644 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2dc8059ba0 con 0x7f2dd80ced80 2024-04-26T19:52:07.142 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.142+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2dd8073c80 con 0x7f2dd80ced80 2024-04-26T19:52:07.143 INFO:tasks.ceph.mon.a.smithi079.stderr:2024-04-26T19:52:07.143+0000 7fbbc119c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T19:52:07.144 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.144+0000 7f2dceffd640 1 -- 172.21.15.79:0/390350644 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f2dc805dea0 con 0x7f2dd80ced80 2024-04-26T19:52:07.145 INFO:teuthology.orchestra.run.smithi079.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T19:52:07.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2db806c740 msgr2=0x7f2db806ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:07.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2db806c740 0x7f2db806ec00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0007fd0 tx=0x7f2dc0007de0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd80ced80 msgr2=0x7f2dd81f0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:07.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd80ced80 0x7f2dd81f0870 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800e9b0 tx=0x7f2dc800ee70 comp rx=0 tx=0).stop 2024-04-26T19:52:07.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 shutdown_connections 2024-04-26T19:52:07.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2db806c740 0x7f2db806ec00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2dd815b850 0x7f2dd8158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2dd80ced80 0x7f2dd81f0870 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.148 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 --2- 172.21.15.79:0/390350644 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2dd8071d50 0x7f2dd806dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 >> 172.21.15.79:0/390350644 conn(0x7f2dd80b39d0 msgr2=0x7f2dd814dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:07.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 shutdown_connections 2024-04-26T19:52:07.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.147+0000 7f2ddcdbf640 1 -- 172.21.15.79:0/390350644 wait complete. 2024-04-26T19:52:07.150 INFO:tasks.ceph.mds.a.smithi079.stderr: -4321> 2024-04-26T19:51:48.801+0000 7f62b1b83640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T19:52:07.162 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T19:52:07.183 INFO:tasks.ceph.mds.a.smithi079.stdout:starting mds.a at 2024-04-26T19:52:07.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.265+0000 7f2eeeb0a640 1 Processor -- start 2024-04-26T19:52:07.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.265+0000 7f2eeeb0a640 1 -- start start 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2eeeb0a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ee815c360 0x7f2ee815c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2eeeb0a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2ee80d0a60 0x7f2ee806fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2eeeb0a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee815cc80 con 0x7f2ee80af9e0 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2eeeb0a640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee806e9b0 con 0x7f2ee80d0a60 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2eeeb0a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee806ec90 con 0x7f2ee815c360 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2ee7fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2ee80d0a60 0x7f2ee806fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2eec8a6640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f2ee815c360 0x7f2ee815c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2ee7fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f2ee80d0a60 0x7f2ee806fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54900/0 (socket says 172.21.15.79:54900) 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2ee7fff640 1 -- 172.21.15.79:0/820207049 learned_addr learned my addr 172.21.15.79:0/820207049 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2eed0a7640 1 --1- 172.21.15.79:0/820207049 >> v1:172.21.15.83:6789/0 conn(0x7f2ee80af9e0 0x7f2ee813ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:58702/0 (socket says 172.21.15.79:58702) 2024-04-26T19:52:07.268 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2ee7fff640 1 -- 172.21.15.79:0/820207049 >> v2:172.21.15.83:3300/0 conn(0x7f2ee815c360 msgr2=0x7f2ee815c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:07.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2ee7fff640 1 --2- 172.21.15.79:0/820207049 >> v2:172.21.15.83:3300/0 conn(0x7f2ee815c360 0x7f2ee815c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2ee7fff640 1 -- 172.21.15.79:0/820207049 >> v1:172.21.15.83:6789/0 conn(0x7f2ee80af9e0 legacy=0x7f2ee813ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:07.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2ee7fff640 1 -- 172.21.15.79:0/820207049 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee806ef70 con 0x7f2ee80d0a60 2024-04-26T19:52:07.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2eec8a6640 1 --2- 172.21.15.79:0/820207049 >> v2:172.21.15.83:3300/0 conn(0x7f2ee815c360 0x7f2ee815c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:07.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.266+0000 7f2ee77fe640 1 -- 172.21.15.79:0/820207049 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 772527199 0 0) 0x7f2ee815cc80 con 0x7f2ee80af9e0 2024-04-26T19:52:07.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.267+0000 7f2ee7fff640 1 --2- 172.21.15.79:0/820207049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 0x7f2ee806fa50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80d1290 tx=0x7f2ed402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75521356e9a0dcbc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:07.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.267+0000 7f2ee77fe640 1 -- 172.21.15.79:0/820207049 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4037070 con 0x7f2ee80d0a60 2024-04-26T19:52:07.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.267+0000 7f2ee77fe640 1 -- 172.21.15.79:0/820207049 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed4032440 con 0x7f2ee80d0a60 2024-04-26T19:52:07.269 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.267+0000 7f2ee77fe640 1 -- 172.21.15.79:0/820207049 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4032810 con 0x7f2ee80d0a60 2024-04-26T19:52:07.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.268+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/820207049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 msgr2=0x7f2ee806fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:07.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.268+0000 7f2eeeb0a640 1 --2- 172.21.15.79:0/820207049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 0x7f2ee806fa50 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2ee80d1290 tx=0x7f2ed402a270 comp rx=0 tx=0).stop 2024-04-26T19:52:07.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.268+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/820207049 shutdown_connections 2024-04-26T19:52:07.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.268+0000 7f2eeeb0a640 1 --2- 172.21.15.79:0/820207049 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 0x7f2ee806fa50 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.268+0000 7f2eeeb0a640 1 --2- 172.21.15.79:0/820207049 >> v2:172.21.15.83:3300/0 conn(0x7f2ee815c360 0x7f2ee815c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.268+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/820207049 >> 172.21.15.79:0/820207049 conn(0x7f2ee80b2670 msgr2=0x7f2ee80b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.268+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/820207049 shutdown_connections 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.268+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/820207049 wait complete. 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.269+0000 7f2eeeb0a640 1 Processor -- start 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.269+0000 7f2eeeb0a640 1 -- start start 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2eeeb0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee80af9e0 0x7f2ee8168640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2eeeb0a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 0x7f2ee80ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2eeeb0a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2ee815c360 0x7f2ee814c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2eeeb0a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee80d0ed0 con 0x7f2ee80d0a60 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2eeeb0a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee806f530 con 0x7f2ee80af9e0 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2eeeb0a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee80d1050 con 0x7f2ee815c360 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 0x7f2ee80ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:07.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2ee7fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 0x7f2ee80ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:54902/0 (socket says 172.21.15.79:54902) 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2ee7fff640 1 -- 172.21.15.79:0/4279184320 learned_addr learned my addr 172.21.15.79:0/4279184320 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2eed0a7640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2ee815c360 0x7f2ee814c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2eec8a6640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee80af9e0 0x7f2ee8168640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2ee7fff640 1 -- 172.21.15.79:0/4279184320 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2ee815c360 msgr2=0x7f2ee814c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2ee7fff640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2ee815c360 0x7f2ee814c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2ee7fff640 1 -- 172.21.15.79:0/4279184320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee80af9e0 msgr2=0x7f2ee8168640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2ee7fff640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee80af9e0 0x7f2ee8168640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.270+0000 7f2ee7fff640 1 -- 172.21.15.79:0/4279184320 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee814bb10 con 0x7f2ee80d0a60 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.271+0000 7f2ee7fff640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 0x7f2ee80ce6a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2ed402a2b0 tx=0x7f2ed40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.271+0000 7f2ee57fa640 1 -- 172.21.15.79:0/4279184320 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4037040 con 0x7f2ee80d0a60 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.271+0000 7f2ee57fa640 1 -- 172.21.15.79:0/4279184320 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ed4042070 con 0x7f2ee80d0a60 2024-04-26T19:52:07.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.271+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee814bdf0 con 0x7f2ee80d0a60 2024-04-26T19:52:07.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.271+0000 7f2ee57fa640 1 -- 172.21.15.79:0/4279184320 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4030880 con 0x7f2ee80d0a60 2024-04-26T19:52:07.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.271+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee8136a90 con 0x7f2ee80d0a60 2024-04-26T19:52:07.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.272+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee80506c0 con 0x7f2ee80d0a60 2024-04-26T19:52:07.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.272+0000 7f2ee57fa640 1 -- 172.21.15.79:0/4279184320 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 13) ==== 90878+0+0 (secure 0 0 0) 0x7f2ed4030a20 con 0x7f2ee80d0a60 2024-04-26T19:52:07.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.273+0000 7f2ee57fa640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2ecc06c740 0x7f2ecc06ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:07.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.273+0000 7f2ee57fa640 1 -- 172.21.15.79:0/4279184320 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6789+0+0 (secure 0 0 0) 0x7f2ed40ac490 con 0x7f2ee80d0a60 2024-04-26T19:52:07.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.273+0000 7f2eec8a6640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2ecc06c740 0x7f2ecc06ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:07.276 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.273+0000 7f2eec8a6640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2ecc06c740 0x7f2ecc06ec00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009a30 tx=0x7f2edc008040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:07.278 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.277+0000 7f2ee57fa640 1 -- 172.21.15.79:0/4279184320 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f2ed407a7c0 con 0x7f2ee80d0a60 2024-04-26T19:52:07.491 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:07.491+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2ee81358d0 con 0x7f2ee80d0a60 2024-04-26T19:52:08.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.145+0000 7f2ee57fa640 1 -- 172.21.15.79:0/4279184320 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7f2ed407a0c0 con 0x7f2ee80d0a60 2024-04-26T19:52:08.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2ecc06c740 msgr2=0x7f2ecc06ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:08.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2ecc06c740 0x7f2ecc06ec00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009a30 tx=0x7f2edc008040 comp rx=0 tx=0).stop 2024-04-26T19:52:08.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 msgr2=0x7f2ee80ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:08.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 0x7f2ee80ce6a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2ed402a2b0 tx=0x7f2ed40046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 shutdown_connections 2024-04-26T19:52:08.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f2ecc06c740 0x7f2ecc06ec00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f2ee815c360 0x7f2ee814c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f2ee80d0a60 0x7f2ee80ce6a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 --2- 172.21.15.79:0/4279184320 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f2ee80af9e0 0x7f2ee8168640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 >> 172.21.15.79:0/4279184320 conn(0x7f2ee80b2670 msgr2=0x7f2ee813ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:08.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 shutdown_connections 2024-04-26T19:52:08.150 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.148+0000 7f2eeeb0a640 1 -- 172.21.15.79:0/4279184320 wait complete. 2024-04-26T19:52:08.163 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T19:52:08.305 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.305+0000 7f3cb7751640 1 Processor -- start 2024-04-26T19:52:08.306 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.305+0000 7f3cb7751640 1 -- start start 2024-04-26T19:52:08.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3cb7751640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3cb006dac0 0x7f3cb006df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:08.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3cb7751640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cb00d2b20 con 0x7f3cb00b7f50 2024-04-26T19:52:08.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3cb7751640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3cb00d2e00 con 0x7f3cb00b4b40 2024-04-26T19:52:08.309 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3cb7751640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3cb00b8330 con 0x7f3cb006dac0 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3cb54ed640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f3cb00b7f50 0x7f3cb00d1380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50866/0 (socket says 172.21.15.79:50866) 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3cb54ed640 1 -- 172.21.15.79:0/1230314090 learned_addr learned my addr 172.21.15.79:0/1230314090 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4180091322 0 0) 0x7f3cb00d2b20 con 0x7f3cb00b7f50 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c9c003520 con 0x7f3cb00b7f50 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3cb4cec640 1 --2- 172.21.15.79:0/1230314090 >> v2:172.21.15.83:3300/0 conn(0x7f3cb006dac0 0x7f3cb006df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1745531570 0 0) 0x7f3c9c003520 con 0x7f3cb00b7f50 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 >> v2:172.21.15.83:3300/0 conn(0x7f3cb006dac0 msgr2=0x7f3cb006df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3ca7fff640 1 --2- 172.21.15.79:0/1230314090 >> v2:172.21.15.83:3300/0 conn(0x7f3cb006dac0 0x7f3cb006df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 >> v1:172.21.15.83:6789/0 conn(0x7f3cb00b4b40 legacy=0x7f3cb00d0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb00d30e0 con 0x7f3cb00b7f50 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f3ca0001690 con 0x7f3cb00b7f50 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3cb4cec640 1 --2- 172.21.15.79:0/1230314090 >> v2:172.21.15.83:3300/0 conn(0x7f3cb006dac0 0x7f3cb006df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.306+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 >> v1:172.21.15.79:6789/0 conn(0x7f3cb00b7f50 legacy=0x7f3cb00d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:08.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3ca7fff640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3c9c0039b0 0x7f3c9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3ca7fff640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9c004800 0x7f3c9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3ca7fff640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3c9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c00c220 con 0x7f3c9c0053c0 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c00c3d0 con 0x7f3c9c004800 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9c00c5d0 con 0x7f3c9c0039b0 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3cb5cee640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3c9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3cb54ed640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3c9c0039b0 0x7f3c9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3cb4cec640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9c004800 0x7f3c9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3cb5cee640 1 -- 172.21.15.79:0/1230314090 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3c9c0039b0 msgr2=0x7f3c9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3cb5cee640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3c9c0039b0 0x7f3c9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3cb5cee640 1 -- 172.21.15.79:0/1230314090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9c004800 msgr2=0x7f3c9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3cb5cee640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3c9c004800 0x7f3c9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.307+0000 7f3cb5cee640 1 -- 172.21.15.79:0/1230314090 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c003520 con 0x7f3c9c0053c0 2024-04-26T19:52:08.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3cb5cee640 1 -- 172.21.15.79:0/1230314090 reap_dead start 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3cb5cee640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3c9c00b9c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8001e50 tx=0x7f3c9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec7cf8e84fa48b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8001a70 con 0x7f3c9c0053c0 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca8003970 con 0x7f3c9c0053c0 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3ca7fff640 1 -- 172.21.15.79:0/1230314090 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8003d60 con 0x7f3c9c0053c0 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3cb7751640 1 -- 172.21.15.79:0/1230314090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 msgr2=0x7f3c9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3cb7751640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3c9c00b9c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8001e50 tx=0x7f3c9c005830 comp rx=0 tx=0).stop 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3cb7751640 1 -- 172.21.15.79:0/1230314090 shutdown_connections 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3cb7751640 1 --2- 172.21.15.79:0/1230314090 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3c9c00b9c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3cb7751640 1 -- 172.21.15.79:0/1230314090 >> 172.21.15.79:0/1230314090 conn(0x7f3cb0071a60 msgr2=0x7f3cb0071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3cb7751640 1 -- 172.21.15.79:0/1230314090 shutdown_connections 2024-04-26T19:52:08.312 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.308+0000 7f3cb7751640 1 -- 172.21.15.79:0/1230314090 wait complete. 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb7751640 1 Processor -- start 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb7751640 1 -- start start 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb7751640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3cb015e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb7751640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb00ce9c0 0x7f3cb00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb7751640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cb00cd8f0 0x7f3cb00cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb7751640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb00b3ea0 con 0x7f3c9c0053c0 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb7751640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb00cfaf0 con 0x7f3cb00ce9c0 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb7751640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb00b4020 con 0x7f3cb00cd8f0 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb54ed640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3cb015e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb54ed640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3cb015e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44076/0 (socket says 172.21.15.79:44076) 2024-04-26T19:52:08.316 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.310+0000 7f3cb54ed640 1 -- 172.21.15.79:0/185331822 learned_addr learned my addr 172.21.15.79:0/185331822 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb5cee640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cb00cd8f0 0x7f3cb00cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb4cec640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb00ce9c0 0x7f3cb00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb54ed640 1 -- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cb00cd8f0 msgr2=0x7f3cb00cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb54ed640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cb00cd8f0 0x7f3cb00cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb54ed640 1 -- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb00ce9c0 msgr2=0x7f3cb00cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb54ed640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb00ce9c0 0x7f3cb00cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb54ed640 1 -- 172.21.15.79:0/185331822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb015e5f0 con 0x7f3c9c0053c0 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb5cee640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cb00cd8f0 0x7f3cb00cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb4cec640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb00ce9c0 0x7f3cb00cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb54ed640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3cb015e0b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0004e90 tx=0x7f3ca0007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3ca5ffb640 1 -- 172.21.15.79:0/185331822 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000f730 con 0x7f3c9c0053c0 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3ca5ffb640 1 -- 172.21.15.79:0/185331822 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca0018070 con 0x7f3c9c0053c0 2024-04-26T19:52:08.317 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb01f6040 con 0x7f3c9c0053c0 2024-04-26T19:52:08.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3ca5ffb640 1 -- 172.21.15.79:0/185331822 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000fb10 con 0x7f3c9c0053c0 2024-04-26T19:52:08.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.311+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb01f6600 con 0x7f3c9c0053c0 2024-04-26T19:52:08.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.312+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb00506c0 con 0x7f3c9c0053c0 2024-04-26T19:52:08.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.312+0000 7f3ca5ffb640 1 -- 172.21.15.79:0/185331822 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 14) ==== 91086+0+0 (secure 0 0 0) 0x7f3ca000fcb0 con 0x7f3c9c0053c0 2024-04-26T19:52:08.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.313+0000 7f3ca5ffb640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3c9406c9b0 0x7f3c9406ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:08.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.313+0000 7f3cb4cec640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3c9406c9b0 0x7f3c9406ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:08.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.313+0000 7f3ca5ffb640 1 -- 172.21.15.79:0/185331822 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6789+0+0 (secure 0 0 0) 0x7f3ca008be50 con 0x7f3c9c0053c0 2024-04-26T19:52:08.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.313+0000 7f3cb4cec640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3c9406c9b0 0x7f3c9406ee70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0070120 tx=0x7f3cac008040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:08.318 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.317+0000 7f3ca5ffb640 1 -- 172.21.15.79:0/185331822 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3ca0059ee0 con 0x7f3c9c0053c0 2024-04-26T19:52:08.492 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:08.491+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3cb00b66e0 con 0x7f3c9c0053c0 2024-04-26T19:52:09.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.207+0000 7f3ca5ffb640 1 -- 172.21.15.79:0/185331822 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7f3ca00597e0 con 0x7f3c9c0053c0 2024-04-26T19:52:09.208 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_metadata' removed 2024-04-26T19:52:09.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3c9406c9b0 msgr2=0x7f3c9406ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:09.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3c9406c9b0 0x7f3c9406ee70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3cb0070120 tx=0x7f3cac008040 comp rx=0 tx=0).stop 2024-04-26T19:52:09.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 msgr2=0x7f3cb015e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:09.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3cb015e0b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f3ca0004e90 tx=0x7f3ca0007430 comp rx=0 tx=0).stop 2024-04-26T19:52:09.211 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 shutdown_connections 2024-04-26T19:52:09.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3c9406c9b0 0x7f3c9406ee70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:09.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3cb00cd8f0 0x7f3cb00cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:09.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3cb00ce9c0 0x7f3cb00cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:09.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 --2- 172.21.15.79:0/185331822 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3c9c0053c0 0x7f3cb015e0b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:09.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 >> 172.21.15.79:0/185331822 conn(0x7f3cb0071a60 msgr2=0x7f3cb015ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:09.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 shutdown_connections 2024-04-26T19:52:09.212 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.211+0000 7f3cb7751640 1 -- 172.21.15.79:0/185331822 wait complete. 2024-04-26T19:52:09.225 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T19:52:09.331 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.331+0000 7f5cb7bc1640 1 Processor -- start 2024-04-26T19:52:09.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.331+0000 7f5cb7bc1640 1 -- start start 2024-04-26T19:52:09.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb7bc1640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5cb0070230 0x7f5cb015db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:09.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb7bc1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb0142f40 con 0x7f5cb0142ad0 2024-04-26T19:52:09.333 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb7bc1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb0161060 con 0x7f5cb01402f0 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb7bc1640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb0161340 con 0x7f5cb0070230 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb515c640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5cb0142ad0 0x7f5cb006fb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50876/0 (socket says 172.21.15.79:50876) 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb515c640 1 -- 172.21.15.79:0/3521090408 learned_addr learned my addr 172.21.15.79:0/3521090408 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb615e640 1 --2- 172.21.15.79:0/3521090408 >> v2:172.21.15.83:3300/0 conn(0x7f5cb0070230 0x7f5cb015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb495b640 1 -- 172.21.15.79:0/3521090408 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3667123751 0 0) 0x7f5cb0142f40 con 0x7f5cb0142ad0 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb495b640 1 -- 172.21.15.79:0/3521090408 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c9c003520 con 0x7f5cb0142ad0 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb495b640 1 -- 172.21.15.79:0/3521090408 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 70145212 0 0) 0x7f5cb0161060 con 0x7f5cb01402f0 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb495b640 1 -- 172.21.15.79:0/3521090408 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cb0142f40 con 0x7f5cb01402f0 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb495b640 1 -- 172.21.15.79:0/3521090408 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3432582490 0 0) 0x7f5c9c003520 con 0x7f5cb0142ad0 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb615e640 1 -- 172.21.15.79:0/3521090408 >> v1:172.21.15.83:6789/0 conn(0x7f5cb01402f0 legacy=0x7f5cb00c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:09.334 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb615e640 1 -- 172.21.15.79:0/3521090408 >> v1:172.21.15.79:6789/0 conn(0x7f5cb0142ad0 legacy=0x7f5cb006fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:09.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.332+0000 7f5cb615e640 1 -- 172.21.15.79:0/3521090408 --> v2:172.21.15.83:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb0161620 con 0x7f5cb0070230 2024-04-26T19:52:09.335 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.333+0000 7f5cb615e640 1 --2- 172.21.15.79:0/3521090408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0070230 0x7f5cb015db10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009d90 tx=0x7f5ca802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38cf60bd8a73acd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:09.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.333+0000 7f5cb495b640 1 -- 172.21.15.79:0/3521090408 <== mon.1 v2:172.21.15.83:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0001670 con 0x7f5cb0070230 2024-04-26T19:52:09.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.333+0000 7f5cb495b640 1 -- 172.21.15.79:0/3521090408 <== mon.1 v2:172.21.15.83:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca8007540 con 0x7f5cb0070230 2024-04-26T19:52:09.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.333+0000 7f5cb495b640 1 -- 172.21.15.79:0/3521090408 <== mon.1 v2:172.21.15.83:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80078f0 con 0x7f5cb0070230 2024-04-26T19:52:09.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.334+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/3521090408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0070230 msgr2=0x7f5cb015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.334+0000 7f5cb7bc1640 1 --2- 172.21.15.79:0/3521090408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0070230 0x7f5cb015db10 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009d90 tx=0x7f5ca802a680 comp rx=0 tx=0).stop 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.334+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/3521090408 shutdown_connections 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.334+0000 7f5cb7bc1640 1 --2- 172.21.15.79:0/3521090408 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0070230 0x7f5cb015db10 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.334+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/3521090408 >> 172.21.15.79:0/3521090408 conn(0x7f5cb006f000 msgr2=0x7f5cb006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.334+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/3521090408 shutdown_connections 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.334+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/3521090408 wait complete. 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.335+0000 7f5cb7bc1640 1 Processor -- start 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.335+0000 7f5cb7bc1640 1 -- start start 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb7bc1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0070230 0x7f5cb013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb7bc1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5cb01402f0 0x7f5cb00cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb7bc1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb0142ad0 0x7f5cb00d1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb7bc1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb01406d0 con 0x7f5cb0142ad0 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb7bc1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb0161be0 con 0x7f5cb0070230 2024-04-26T19:52:09.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb7bc1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb00c8eb0 con 0x7f5cb01402f0 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb615e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb0142ad0 0x7f5cb00d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb615e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb0142ad0 0x7f5cb00d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44082/0 (socket says 172.21.15.79:44082) 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb615e640 1 -- 172.21.15.79:0/2464059545 learned_addr learned my addr 172.21.15.79:0/2464059545 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb515c640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5cb01402f0 0x7f5cb00cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb595d640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0070230 0x7f5cb013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb615e640 1 -- 172.21.15.79:0/2464059545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5cb01402f0 msgr2=0x7f5cb00cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb615e640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5cb01402f0 0x7f5cb00cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb615e640 1 -- 172.21.15.79:0/2464059545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0070230 msgr2=0x7f5cb013cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb615e640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0070230 0x7f5cb013cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb615e640 1 -- 172.21.15.79:0/2464059545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb013d3e0 con 0x7f5cb0142ad0 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.336+0000 7f5cb615e640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb0142ad0 0x7f5cb00d1590 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5ca802ac70 tx=0x7f5ca8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.337+0000 7f5ca67fc640 1 -- 172.21.15.79:0/2464059545 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8009450 con 0x7f5cb0142ad0 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.337+0000 7f5ca67fc640 1 -- 172.21.15.79:0/2464059545 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca8004240 con 0x7f5cb0142ad0 2024-04-26T19:52:09.340 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.337+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb00cec60 con 0x7f5cb0142ad0 2024-04-26T19:52:09.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.337+0000 7f5ca67fc640 1 -- 172.21.15.79:0/2464059545 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca80045f0 con 0x7f5cb0142ad0 2024-04-26T19:52:09.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.337+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb00cf200 con 0x7f5cb0142ad0 2024-04-26T19:52:09.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.338+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb00506c0 con 0x7f5cb0142ad0 2024-04-26T19:52:09.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.338+0000 7f5ca67fc640 1 -- 172.21.15.79:0/2464059545 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 14) ==== 91086+0+0 (secure 0 0 0) 0x7f5ca803b410 con 0x7f5cb0142ad0 2024-04-26T19:52:09.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.338+0000 7f5ca67fc640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5c9406c980 0x7f5c9406ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:09.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.338+0000 7f5ca67fc640 1 -- 172.21.15.79:0/2464059545 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6351+0+0 (secure 0 0 0) 0x7f5ca80ad120 con 0x7f5cb0142ad0 2024-04-26T19:52:09.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.338+0000 7f5cb595d640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5c9406c980 0x7f5c9406ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:09.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.339+0000 7f5cb595d640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5c9406c980 0x7f5c9406ee40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0011e0 tx=0x7f5cac009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:09.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.343+0000 7f5ca67fc640 1 -- 172.21.15.79:0/2464059545 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5cb013d3e0 con 0x7f5cb0142ad0 2024-04-26T19:52:09.517 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.517+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5cb006c810 con 0x7f5cb0142ad0 2024-04-26T19:52:09.609 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:09.608+0000 7f5ca67fc640 1 -- 172.21.15.79:0/2464059545 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f5ca80771d0 con 0x7f5cb0142ad0 2024-04-26T19:52:10.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.209+0000 7f5ca67fc640 1 -- 172.21.15.79:0/2464059545 <== mon.0 v2:172.21.15.79:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7f5ca803f030 con 0x7f5cb0142ad0 2024-04-26T19:52:10.210 INFO:teuthology.orchestra.run.smithi079.stderr:pool 'cephfs_data' removed 2024-04-26T19:52:10.213 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.212+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5c9406c980 msgr2=0x7f5c9406ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.212+0000 7f5cb7bc1640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5c9406c980 0x7f5c9406ee40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0011e0 tx=0x7f5cac009040 comp rx=0 tx=0).stop 2024-04-26T19:52:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.212+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb0142ad0 msgr2=0x7f5cb00d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.212+0000 7f5cb7bc1640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb0142ad0 0x7f5cb00d1590 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5ca802ac70 tx=0x7f5ca8004a40 comp rx=0 tx=0).stop 2024-04-26T19:52:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.213+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 shutdown_connections 2024-04-26T19:52:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.213+0000 7f5cb7bc1640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5c9406c980 0x7f5c9406ee40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.213+0000 7f5cb7bc1640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5cb0142ad0 0x7f5cb00d1590 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.213+0000 7f5cb7bc1640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5cb01402f0 0x7f5cb00cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.213+0000 7f5cb7bc1640 1 --2- 172.21.15.79:0/2464059545 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5cb0070230 0x7f5cb013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.214 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.213+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 >> 172.21.15.79:0/2464059545 conn(0x7f5cb006f000 msgr2=0x7f5cb00c7920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:10.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.213+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 shutdown_connections 2024-04-26T19:52:10.215 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.213+0000 7f5cb7bc1640 1 -- 172.21.15.79:0/2464059545 wait complete. 2024-04-26T19:52:10.228 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-26T19:52:10.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.331+0000 7f38e302a640 1 Processor -- start 2024-04-26T19:52:10.332 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.331+0000 7f38e302a640 1 -- start start 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38e302a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f38dc15ced0 0x7f38dc1638c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38e302a640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38dc160380 con 0x7f38dc0ab280 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38e302a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38dc160660 con 0x7f38dc0d5220 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38e302a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f38dc1630c0 con 0x7f38dc15ced0 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38e15c7640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f38dc0ab280 0x7f38dc0ab6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50888/0 (socket says 172.21.15.79:50888) 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38e15c7640 1 -- 172.21.15.79:0/2318729845 learned_addr learned my addr 172.21.15.79:0/2318729845 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2408599593 0 0) 0x7f38dc160380 con 0x7f38dc0ab280 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38cc003520 con 0x7f38dc0ab280 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38e0dc6640 1 --2- 172.21.15.79:0/2318729845 >> v2:172.21.15.83:3300/0 conn(0x7f38dc15ced0 0x7f38dc1638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3341580106 0 0) 0x7f38cc003520 con 0x7f38dc0ab280 2024-04-26T19:52:10.336 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 >> v2:172.21.15.83:3300/0 conn(0x7f38dc15ced0 msgr2=0x7f38dc1638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 --2- 172.21.15.79:0/2318729845 >> v2:172.21.15.83:3300/0 conn(0x7f38dc15ced0 0x7f38dc1638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 >> v1:172.21.15.83:6789/0 conn(0x7f38dc0d5220 legacy=0x7f38dc15f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc1633a0 con 0x7f38dc0ab280 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f38d40019f0 con 0x7f38dc0ab280 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 >> v1:172.21.15.79:6789/0 conn(0x7f38dc0ab280 legacy=0x7f38dc0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0039b0 0x7f38cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.332+0000 7f38db7fe640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f38cc004800 0x7f38cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38db7fe640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0051c0 0x7f38cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc1630c0 con 0x7f38cc0051c0 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38cc00c060 con 0x7f38cc0039b0 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f38cc00c260 con 0x7f38cc004800 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38e15c7640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0051c0 0x7f38cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38e0dc6640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0039b0 0x7f38cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.337 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38dbfff640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f38cc004800 0x7f38cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38e15c7640 1 -- 172.21.15.79:0/2318729845 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f38cc004800 msgr2=0x7f38cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38e15c7640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f38cc004800 0x7f38cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38e15c7640 1 -- 172.21.15.79:0/2318729845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0039b0 msgr2=0x7f38cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38e15c7640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0039b0 0x7f38cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38e15c7640 1 -- 172.21.15.79:0/2318729845 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38cc003520 con 0x7f38cc0051c0 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.333+0000 7f38e15c7640 1 -- 172.21.15.79:0/2318729845 reap_dead start 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38e15c7640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0051c0 0x7f38cc00b9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f38d400a840 tx=0x7f38cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146746bfbf0f0a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d400a460 con 0x7f38cc0051c0 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d402f070 con 0x7f38cc0051c0 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38db7fe640 1 -- 172.21.15.79:0/2318729845 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d402bcb0 con 0x7f38cc0051c0 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38e302a640 1 -- 172.21.15.79:0/2318729845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0051c0 msgr2=0x7f38cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38e302a640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0051c0 0x7f38cc00b9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f38d400a840 tx=0x7f38cc005830 comp rx=0 tx=0).stop 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38e302a640 1 -- 172.21.15.79:0/2318729845 shutdown_connections 2024-04-26T19:52:10.338 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38e302a640 1 --2- 172.21.15.79:0/2318729845 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38cc0051c0 0x7f38cc00b9c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38e302a640 1 -- 172.21.15.79:0/2318729845 >> 172.21.15.79:0/2318729845 conn(0x7f38dc0687b0 msgr2=0x7f38dc071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:10.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38e302a640 1 -- 172.21.15.79:0/2318729845 shutdown_connections 2024-04-26T19:52:10.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.334+0000 7f38e302a640 1 -- 172.21.15.79:0/2318729845 wait complete. 2024-04-26T19:52:10.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.335+0000 7f38e302a640 1 Processor -- start 2024-04-26T19:52:10.339 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.335+0000 7f38e302a640 1 -- start start 2024-04-26T19:52:10.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.336+0000 7f38e302a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0051c0 0x7f38dc155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.341 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.336+0000 7f38e302a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f38dc0d2670 0x7f38dc1f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.336+0000 7f38e302a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38dc0d2af0 0x7f38dc1f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.336+0000 7f38e302a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc0ab820 con 0x7f38dc0d2af0 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.336+0000 7f38e302a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc0ab6a0 con 0x7f38cc0051c0 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.336+0000 7f38e302a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f38dc0ab9a0 con 0x7f38dc0d2670 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.336+0000 7f38e15c7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38dc0d2af0 0x7f38dc1f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.336+0000 7f38e15c7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38dc0d2af0 0x7f38dc1f92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44110/0 (socket says 172.21.15.79:44110) 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.336+0000 7f38e15c7640 1 -- 172.21.15.79:0/650270754 learned_addr learned my addr 172.21.15.79:0/650270754 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e0dc6640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0051c0 0x7f38dc155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38dbfff640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f38dc0d2670 0x7f38dc1f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e15c7640 1 -- 172.21.15.79:0/650270754 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f38dc0d2670 msgr2=0x7f38dc1f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e15c7640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f38dc0d2670 0x7f38dc1f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e15c7640 1 -- 172.21.15.79:0/650270754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0051c0 msgr2=0x7f38dc155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e15c7640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0051c0 0x7f38dc155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.342 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e15c7640 1 -- 172.21.15.79:0/650270754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc157f30 con 0x7f38dc0d2af0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e0dc6640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0051c0 0x7f38dc155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e15c7640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38dc0d2af0 0x7f38dc1f92c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f38d402b500 tx=0x7f38dc160550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38d97fa640 1 -- 172.21.15.79:0/650270754 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4001fb0 con 0x7f38dc0d2af0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38d97fa640 1 -- 172.21.15.79:0/650270754 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d4004d60 con 0x7f38dc0d2af0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc158210 con 0x7f38dc0d2af0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38d97fa640 1 -- 172.21.15.79:0/650270754 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4034670 con 0x7f38dc0d2af0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.337+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc1f9800 con 0x7f38dc0d2af0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.338+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc0506c0 con 0x7f38dc0d2af0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.339+0000 7f38d97fa640 1 -- 172.21.15.79:0/650270754 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f38d4034810 con 0x7f38dc0d2af0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.339+0000 7f38d97fa640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f38c006ca10 0x7f38c006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.339+0000 7f38d97fa640 1 -- 172.21.15.79:0/650270754 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f38d40abdc0 con 0x7f38dc0d2af0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.339+0000 7f38e0dc6640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f38c006ca10 0x7f38c006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.343 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.339+0000 7f38e0dc6640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f38c006ca10 0x7f38c006eed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009b40 tx=0x7f38d0008040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:10.344 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.343+0000 7f38d97fa640 1 -- 172.21.15.79:0/650270754 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f38d407a080 con 0x7f38dc0d2af0 2024-04-26T19:52:10.520 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.519+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f38dc160830 con 0x7f38dc0d2af0 2024-04-26T19:52:10.522 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.522+0000 7f38d97fa640 1 -- 172.21.15.79:0/650270754 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7f38d4079980 con 0x7f38dc0d2af0 2024-04-26T19:52:10.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f38c006ca10 msgr2=0x7f38c006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f38c006ca10 0x7f38c006eed0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38d0009b40 tx=0x7f38d0008040 comp rx=0 tx=0).stop 2024-04-26T19:52:10.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38dc0d2af0 msgr2=0x7f38dc1f92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38dc0d2af0 0x7f38dc1f92c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f38d402b500 tx=0x7f38dc160550 comp rx=0 tx=0).stop 2024-04-26T19:52:10.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 shutdown_connections 2024-04-26T19:52:10.525 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f38c006ca10 0x7f38c006eed0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38dc0d2af0 0x7f38dc1f92c0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f38dc0d2670 0x7f38dc1f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 --2- 172.21.15.79:0/650270754 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38cc0051c0 0x7f38dc155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 >> 172.21.15.79:0/650270754 conn(0x7f38dc0687b0 msgr2=0x7f38dc15de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:10.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 shutdown_connections 2024-04-26T19:52:10.526 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.524+0000 7f38e302a640 1 -- 172.21.15.79:0/650270754 wait complete. 2024-04-26T19:52:10.536 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2024-04-26T19:52:10.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.633+0000 7f16ca6d1640 1 Processor -- start 2024-04-26T19:52:10.634 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.633+0000 7f16ca6d1640 1 -- start start 2024-04-26T19:52:10.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.634+0000 7f16ca6d1640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f16c40d1ac0 0x7f16c40b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.634+0000 7f16ca6d1640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16c40d3070 con 0x7f16c415f770 2024-04-26T19:52:10.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.634+0000 7f16ca6d1640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16c406cc70 con 0x7f16c40d1f20 2024-04-26T19:52:10.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.634+0000 7f16ca6d1640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f16c406cf50 con 0x7f16c40d1ac0 2024-04-26T19:52:10.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.634+0000 7f16c37fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f16c40d1ac0 0x7f16c40b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.634+0000 7f16c3fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f16c415f770 0x7f16c40d13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50890/0 (socket says 172.21.15.79:50890) 2024-04-26T19:52:10.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.634+0000 7f16c3fff640 1 -- 172.21.15.79:0/1879209444 learned_addr learned my addr 172.21.15.79:0/1879209444 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:10.636 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.635+0000 7f16c37fe640 1 -- 172.21.15.79:0/1879209444 >> v1:172.21.15.83:6789/0 conn(0x7f16c40d1f20 legacy=0x7f16c40b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:10.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.635+0000 7f16c37fe640 1 -- 172.21.15.79:0/1879209444 >> v1:172.21.15.79:6789/0 conn(0x7f16c415f770 legacy=0x7f16c40d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.635+0000 7f16c2ffd640 1 -- 172.21.15.79:0/1879209444 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3668428755 0 0) 0x7f16c40d3070 con 0x7f16c415f770 2024-04-26T19:52:10.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.635+0000 7f16c37fe640 1 -- 172.21.15.79:0/1879209444 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c406d230 con 0x7f16c40d1ac0 2024-04-26T19:52:10.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.635+0000 7f16c37fe640 1 --2- 172.21.15.79:0/1879209444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1ac0 0x7f16c40b2d50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f16b80097a0 tx=0x7f16b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e784f3ebf70ecb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:10.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.635+0000 7f16c2ffd640 1 -- 172.21.15.79:0/1879209444 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8037070 con 0x7f16c40d1ac0 2024-04-26T19:52:10.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.635+0000 7f16c2ffd640 1 -- 172.21.15.79:0/1879209444 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b8032440 con 0x7f16c40d1ac0 2024-04-26T19:52:10.637 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.635+0000 7f16c2ffd640 1 -- 172.21.15.79:0/1879209444 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8032810 con 0x7f16c40d1ac0 2024-04-26T19:52:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.636+0000 7f16ca6d1640 1 -- 172.21.15.79:0/1879209444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1ac0 msgr2=0x7f16c40b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.636+0000 7f16ca6d1640 1 --2- 172.21.15.79:0/1879209444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1ac0 0x7f16c40b2d50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f16b80097a0 tx=0x7f16b802a1e0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.636+0000 7f16ca6d1640 1 -- 172.21.15.79:0/1879209444 shutdown_connections 2024-04-26T19:52:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.636+0000 7f16ca6d1640 1 --2- 172.21.15.79:0/1879209444 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1ac0 0x7f16c40b2d50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.636+0000 7f16ca6d1640 1 -- 172.21.15.79:0/1879209444 >> 172.21.15.79:0/1879209444 conn(0x7f16c40b4500 msgr2=0x7f16c40b48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.636+0000 7f16ca6d1640 1 -- 172.21.15.79:0/1879209444 shutdown_connections 2024-04-26T19:52:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.636+0000 7f16ca6d1640 1 -- 172.21.15.79:0/1879209444 wait complete. 2024-04-26T19:52:10.641 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.637+0000 7f16ca6d1640 1 Processor -- start 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.637+0000 7f16ca6d1640 1 -- start start 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16ca6d1640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f16c40d1ac0 0x7f16c414bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16ca6d1640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1f20 0x7f16c406f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16ca6d1640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16c415f770 0x7f16c406fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16ca6d1640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f16c40b69d0 con 0x7f16c40d1f20 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16ca6d1640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f16c40b6850 con 0x7f16c415f770 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16ca6d1640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f16c40d2c10 con 0x7f16c40d1ac0 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1f20 0x7f16c406f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c37fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1f20 0x7f16c406f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44142/0 (socket says 172.21.15.79:44142) 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c37fe640 1 -- 172.21.15.79:0/2301006016 learned_addr learned my addr 172.21.15.79:0/2301006016 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:10.642 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c8c6e640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16c415f770 0x7f16c406fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c3fff640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f16c40d1ac0 0x7f16c414bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c37fe640 1 -- 172.21.15.79:0/2301006016 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f16c40d1ac0 msgr2=0x7f16c414bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c37fe640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f16c40d1ac0 0x7f16c414bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c37fe640 1 -- 172.21.15.79:0/2301006016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16c415f770 msgr2=0x7f16c406fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c37fe640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16c415f770 0x7f16c406fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c37fe640 1 -- 172.21.15.79:0/2301006016 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c415ba70 con 0x7f16c40d1f20 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.638+0000 7f16c3fff640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f16c40d1ac0 0x7f16c414bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.639+0000 7f16c37fe640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1f20 0x7f16c406f530 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f16b80097a0 tx=0x7f16b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.639+0000 7f16c0ff9640 1 -- 172.21.15.79:0/2301006016 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8037040 con 0x7f16c40d1f20 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.639+0000 7f16c0ff9640 1 -- 172.21.15.79:0/2301006016 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b8004250 con 0x7f16c40d1f20 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.639+0000 7f16c0ff9640 1 -- 172.21.15.79:0/2301006016 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b8032e20 con 0x7f16c40d1f20 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.639+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c415bd10 con 0x7f16c40d1f20 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.639+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c4161370 con 0x7f16c40d1f20 2024-04-26T19:52:10.643 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.640+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c40506c0 con 0x7f16c40d1f20 2024-04-26T19:52:10.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.640+0000 7f16c0ff9640 1 -- 172.21.15.79:0/2301006016 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f16b8030770 con 0x7f16c40d1f20 2024-04-26T19:52:10.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.640+0000 7f16c0ff9640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f16b006ca10 0x7f16b006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.641+0000 7f16c0ff9640 1 -- 172.21.15.79:0/2301006016 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f16b80ad110 con 0x7f16c40d1f20 2024-04-26T19:52:10.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.641+0000 7f16c3fff640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f16b006ca10 0x7f16b006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.644 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.641+0000 7f16c3fff640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f16b006ca10 0x7f16b006eed0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16b4000f60 tx=0x7f16b4009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:10.645 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.645+0000 7f16c0ff9640 1 -- 172.21.15.79:0/2301006016 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f16b807b340 con 0x7f16c40d1f20 2024-04-26T19:52:10.819 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.819+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) -- 0x7f16c40d29f0 con 0x7f16c40d1f20 2024-04-26T19:52:10.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.822+0000 7f16c0ff9640 1 -- 172.21.15.79:0/2301006016 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) ==== 115+0+0 (secure 0 0 0) 0x7f16b807ac40 con 0x7f16c40d1f20 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f16b006ca10 msgr2=0x7f16b006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f16b006ca10 0x7f16b006eed0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f16b4000f60 tx=0x7f16b4009040 comp rx=0 tx=0).stop 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1f20 msgr2=0x7f16c406f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1f20 0x7f16c406f530 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f16b80097a0 tx=0x7f16b80046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 shutdown_connections 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f16b006ca10 0x7f16b006eed0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f16c415f770 0x7f16c406fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f16c40d1f20 0x7f16c406f530 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 --2- 172.21.15.79:0/2301006016 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f16c40d1ac0 0x7f16c414bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 >> 172.21.15.79:0/2301006016 conn(0x7f16c40b4500 msgr2=0x7f16c40b7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:10.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 shutdown_connections 2024-04-26T19:52:10.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.825+0000 7f16ca6d1640 1 -- 172.21.15.79:0/2301006016 wait complete. 2024-04-26T19:52:10.837 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2024-04-26T19:52:10.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.940+0000 7fc58d220640 1 Processor -- start 2024-04-26T19:52:10.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.940+0000 7fc58d220640 1 -- start start 2024-04-26T19:52:10.951 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58d220640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fc5881423c0 0x7fc588142c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58d220640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc588073710 0x7fc5880b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58d220640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 0x7fc58815fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58d220640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fc588143180 con 0x7fc588073710 2024-04-26T19:52:10.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58d220640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fc588163fa0 con 0x7fc5881423c0 2024-04-26T19:52:10.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58d220640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc588163420 con 0x7fc5880b4230 2024-04-26T19:52:10.952 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58659c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc588073710 0x7fc5880b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58659c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fc588073710 0x7fc5880b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44144/0 (socket says 172.21.15.79:44144) 2024-04-26T19:52:10.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58659c640 1 -- 172.21.15.79:0/2896849775 learned_addr learned my addr 172.21.15.79:0/2896849775 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:10.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58759e640 1 --2- 172.21.15.79:0/2896849775 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 0x7fc58815fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc586d9d640 1 --2- 172.21.15.79:0/2896849775 >> v2:172.21.15.83:3300/0 conn(0x7fc5881423c0 0x7fc588142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58659c640 1 -- 172.21.15.79:0/2896849775 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 msgr2=0x7fc58815fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:10.953 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58659c640 1 --2- 172.21.15.79:0/2896849775 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 0x7fc58815fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58659c640 1 -- 172.21.15.79:0/2896849775 >> v2:172.21.15.83:3300/0 conn(0x7fc5881423c0 msgr2=0x7fc588142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:10.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58659c640 1 --2- 172.21.15.79:0/2896849775 >> v2:172.21.15.83:3300/0 conn(0x7fc5881423c0 0x7fc588142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.941+0000 7fc58659c640 1 -- 172.21.15.79:0/2896849775 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc588163700 con 0x7fc588073710 2024-04-26T19:52:10.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.942+0000 7fc58659c640 1 --2- 172.21.15.79:0/2896849775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc588073710 0x7fc5880b3cf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc5880b3840 tx=0x7fc57c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a22b603d165705 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:10.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.942+0000 7fc585d9b640 1 -- 172.21.15.79:0/2896849775 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c037070 con 0x7fc588073710 2024-04-26T19:52:10.954 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.942+0000 7fc585d9b640 1 -- 172.21.15.79:0/2896849775 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc57c032440 con 0x7fc588073710 2024-04-26T19:52:10.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.942+0000 7fc585d9b640 1 -- 172.21.15.79:0/2896849775 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c032810 con 0x7fc588073710 2024-04-26T19:52:10.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.943+0000 7fc58d220640 1 -- 172.21.15.79:0/2896849775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc588073710 msgr2=0x7fc5880b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.943+0000 7fc58d220640 1 --2- 172.21.15.79:0/2896849775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc588073710 0x7fc5880b3cf0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc5880b3840 tx=0x7fc57c02a1e0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.943+0000 7fc58d220640 1 -- 172.21.15.79:0/2896849775 shutdown_connections 2024-04-26T19:52:10.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.943+0000 7fc58d220640 1 --2- 172.21.15.79:0/2896849775 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 0x7fc58815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.943+0000 7fc58d220640 1 --2- 172.21.15.79:0/2896849775 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc588073710 0x7fc5880b3cf0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.955 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.943+0000 7fc58d220640 1 --2- 172.21.15.79:0/2896849775 >> v2:172.21.15.83:3300/0 conn(0x7fc5881423c0 0x7fc588142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.943+0000 7fc58d220640 1 -- 172.21.15.79:0/2896849775 >> 172.21.15.79:0/2896849775 conn(0x7fc5880b2990 msgr2=0x7fc5880b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:10.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.943+0000 7fc58d220640 1 -- 172.21.15.79:0/2896849775 shutdown_connections 2024-04-26T19:52:10.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.943+0000 7fc58d220640 1 -- 172.21.15.79:0/2896849775 wait complete. 2024-04-26T19:52:10.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.944+0000 7fc58d220640 1 Processor -- start 2024-04-26T19:52:10.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.944+0000 7fc58d220640 1 -- start start 2024-04-26T19:52:10.956 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58d220640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc588073710 0x7fc58813b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58d220640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 0x7fc588161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58d220640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5881423c0 0x7fc5881395a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58d220640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fc58806e8b0 con 0x7fc5881423c0 2024-04-26T19:52:10.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58d220640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fc588163cc0 con 0x7fc588073710 2024-04-26T19:52:10.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58d220640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fc588073ba0 con 0x7fc5880b4230 2024-04-26T19:52:10.957 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58759e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5881423c0 0x7fc5881395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58759e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5881423c0 0x7fc5881395a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44160/0 (socket says 172.21.15.79:44160) 2024-04-26T19:52:10.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58759e640 1 -- 172.21.15.79:0/2219276795 learned_addr learned my addr 172.21.15.79:0/2219276795 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:10.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58659c640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 0x7fc588161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc586d9d640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc588073710 0x7fc58813b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.958 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58759e640 1 -- 172.21.15.79:0/2219276795 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 msgr2=0x7fc588161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58759e640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 0x7fc588161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58759e640 1 -- 172.21.15.79:0/2219276795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc588073710 msgr2=0x7fc58813b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:10.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58759e640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc588073710 0x7fc58813b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:10.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58759e640 1 -- 172.21.15.79:0/2219276795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc588160a00 con 0x7fc5881423c0 2024-04-26T19:52:10.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc586d9d640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc588073710 0x7fc58813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:10.959 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.945+0000 7fc58759e640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5881423c0 0x7fc5881395a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc58000d800 tx=0x7fc58000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:10.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.946+0000 7fc56f7fe640 1 -- 172.21.15.79:0/2219276795 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc580018020 con 0x7fc5881423c0 2024-04-26T19:52:10.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.946+0000 7fc56f7fe640 1 -- 172.21.15.79:0/2219276795 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5800096c0 con 0x7fc5881423c0 2024-04-26T19:52:10.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.946+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc588160ce0 con 0x7fc5881423c0 2024-04-26T19:52:10.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.946+0000 7fc56f7fe640 1 -- 172.21.15.79:0/2219276795 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc580009a90 con 0x7fc5881423c0 2024-04-26T19:52:10.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.946+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5881612a0 con 0x7fc5881423c0 2024-04-26T19:52:10.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.947+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5880506c0 con 0x7fc5881423c0 2024-04-26T19:52:10.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.947+0000 7fc56f7fe640 1 -- 172.21.15.79:0/2219276795 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7fc58001b020 con 0x7fc5881423c0 2024-04-26T19:52:10.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.947+0000 7fc56f7fe640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fc57406ca10 0x7fc57406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:10.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.948+0000 7fc56f7fe640 1 -- 172.21.15.79:0/2219276795 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fc58008bf90 con 0x7fc5881423c0 2024-04-26T19:52:10.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.948+0000 7fc586d9d640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fc57406ca10 0x7fc57406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:10.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.948+0000 7fc586d9d640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fc57406ca10 0x7fc57406eed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc578009a80 tx=0x7fc5780039d0 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:10.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:10.952+0000 7fc56f7fe640 1 -- 172.21.15.79:0/2219276795 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fc580022030 con 0x7fc5881423c0 2024-04-26T19:52:11.125 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.124+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) -- 0x7fc5880d1090 con 0x7fc5881423c0 2024-04-26T19:52:11.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.127+0000 7fc56f7fe640 1 -- 172.21.15.79:0/2219276795 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7fc58005a1b0 con 0x7fc5881423c0 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fc57406ca10 msgr2=0x7fc57406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fc57406ca10 0x7fc57406eed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc578009a80 tx=0x7fc5780039d0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5881423c0 msgr2=0x7fc5881395a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5881423c0 0x7fc5881395a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc58000d800 tx=0x7fc58000dcc0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 shutdown_connections 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fc57406ca10 0x7fc57406eed0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fc5881423c0 0x7fc5881395a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fc5880b4230 0x7fc588161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 --2- 172.21.15.79:0/2219276795 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fc588073710 0x7fc58813b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 >> 172.21.15.79:0/2219276795 conn(0x7fc5880b2990 msgr2=0x7fc588142a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:11.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 shutdown_connections 2024-04-26T19:52:11.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.130+0000 7fc58d220640 1 -- 172.21.15.79:0/2219276795 wait complete. 2024-04-26T19:52:11.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2024-04-26T19:52:11.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.240+0000 7f7f517f7640 1 Processor -- start 2024-04-26T19:52:11.241 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.241+0000 7f7f517f7640 1 -- start start 2024-04-26T19:52:11.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.241+0000 7f7f517f7640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f7f4c06ff00 0x7f7f4c148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:11.242 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.241+0000 7f7f517f7640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7f4c0d2cd0 0x7f7f4c0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.241+0000 7f7f517f7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f4c0b1290 con 0x7f7f4c06fb20 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.241+0000 7f7f517f7640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4c0d3140 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.241+0000 7f7f517f7640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f7f4c0d3e60 con 0x7f7f4c06ff00 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.241+0000 7f7f4b7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7f4c0d2cd0 0x7f7f4c0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.241+0000 7f7f4b7fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f7f4c0d2cd0 0x7f7f4c0d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44168/0 (socket says 172.21.15.79:44168) 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.241+0000 7f7f4b7fe640 1 -- 172.21.15.79:0/2401817864 learned_addr learned my addr 172.21.15.79:0/2401817864 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.242+0000 7f7f4a7fc640 1 --2- 172.21.15.79:0/2401817864 >> v2:172.21.15.83:3300/0 conn(0x7f7f4c06ff00 0x7f7f4c148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.242+0000 7f7f4b7fe640 1 -- 172.21.15.79:0/2401817864 >> v2:172.21.15.83:3300/0 conn(0x7f7f4c06ff00 msgr2=0x7f7f4c148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.242+0000 7f7f4b7fe640 1 --2- 172.21.15.79:0/2401817864 >> v2:172.21.15.83:3300/0 conn(0x7f7f4c06ff00 0x7f7f4c148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.242+0000 7f7f4b7fe640 1 -- 172.21.15.79:0/2401817864 >> v1:172.21.15.83:6789/0 conn(0x7f7f4c06fb20 legacy=0x7f7f4c1488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:11.243 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.242+0000 7f7f4b7fe640 1 -- 172.21.15.79:0/2401817864 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c0d4090 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.242+0000 7f7f4b7fe640 1 --2- 172.21.15.79:0/2401817864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 0x7f7f4c0d3920 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c149540 tx=0x7f7f4402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d7c730cd4daa14f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:11.244 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f49ffb640 1 -- 172.21.15.79:0/2401817864 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44037070 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f49ffb640 1 -- 172.21.15.79:0/2401817864 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4402ae50 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f49ffb640 1 -- 172.21.15.79:0/2401817864 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44032860 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f517f7640 1 -- 172.21.15.79:0/2401817864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 msgr2=0x7f7f4c0d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:11.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f517f7640 1 --2- 172.21.15.79:0/2401817864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 0x7f7f4c0d3920 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c149540 tx=0x7f7f4402a300 comp rx=0 tx=0).stop 2024-04-26T19:52:11.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f517f7640 1 -- 172.21.15.79:0/2401817864 shutdown_connections 2024-04-26T19:52:11.245 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f517f7640 1 --2- 172.21.15.79:0/2401817864 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 0x7f7f4c0d3920 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f517f7640 1 --2- 172.21.15.79:0/2401817864 >> v2:172.21.15.83:3300/0 conn(0x7f7f4c06ff00 0x7f7f4c148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f517f7640 1 -- 172.21.15.79:0/2401817864 >> 172.21.15.79:0/2401817864 conn(0x7f7f4c071c30 msgr2=0x7f7f4c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:11.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f517f7640 1 -- 172.21.15.79:0/2401817864 shutdown_connections 2024-04-26T19:52:11.246 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.243+0000 7f7f517f7640 1 -- 172.21.15.79:0/2401817864 wait complete. 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.244+0000 7f7f517f7640 1 Processor -- start 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f517f7640 1 -- start start 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f517f7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c06fb20 0x7f7f4c0b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f517f7640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7f4c06ff00 0x7f7f4c1f7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f517f7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 0x7f7f4c0b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f517f7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c0d4930 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f517f7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c0d4650 con 0x7f7f4c06fb20 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f517f7640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f7f4c0d4c10 con 0x7f7f4c06ff00 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 0x7f7f4c0b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:11.251 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f4b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 0x7f7f4c0b2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44178/0 (socket says 172.21.15.79:44178) 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f4b7fe640 1 -- 172.21.15.79:0/2091377935 learned_addr learned my addr 172.21.15.79:0/2091377935 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f4a7fc640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7f4c06ff00 0x7f7f4c1f7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.245+0000 7f7f4affd640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c06fb20 0x7f7f4c0b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f4b7fe640 1 -- 172.21.15.79:0/2091377935 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7f4c06ff00 msgr2=0x7f7f4c1f7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f4b7fe640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7f4c06ff00 0x7f7f4c1f7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f4b7fe640 1 -- 172.21.15.79:0/2091377935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c06fb20 msgr2=0x7f7f4c0b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f4b7fe640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c06fb20 0x7f7f4c0b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f4b7fe640 1 -- 172.21.15.79:0/2091377935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c0b2030 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f4b7fe640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 0x7f7f4c0b2df0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7f4402a340 tx=0x7f7f440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f337fe640 1 -- 172.21.15.79:0/2091377935 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44041020 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f337fe640 1 -- 172.21.15.79:0/2091377935 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f4403c890 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c0b2310 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f337fe640 1 -- 172.21.15.79:0/2091377935 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44037040 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.246+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c0b28d0 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.252 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.247+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4c0506c0 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.247+0000 7f7f337fe640 1 -- 172.21.15.79:0/2091377935 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f7f44044020 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.248+0000 7f7f337fe640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f7f3806ca10 0x7f7f3806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:11.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.248+0000 7f7f4affd640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f7f3806ca10 0x7f7f3806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:11.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.248+0000 7f7f337fe640 1 -- 172.21.15.79:0/2091377935 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f7f440ac570 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.248+0000 7f7f4affd640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f7f3806ca10 0x7f7f3806eed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c001f10 tx=0x7f7f3c009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:11.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.252+0000 7f7f337fe640 1 -- 172.21.15.79:0/2091377935 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f7f4404b030 con 0x7f7f4c0d2cd0 2024-04-26T19:52:11.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:11.497+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) -- 0x7f7f4c072a50 con 0x7f7f4c0d2cd0 2024-04-26T19:52:12.218 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.217+0000 7f7f337fe640 1 -- 172.21.15.79:0/2091377935 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v118) ==== 155+0+0 (secure 0 0 0) 0x7f7f4407a150 con 0x7f7f4c0d2cd0 2024-04-26T19:52:12.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f7f3806ca10 msgr2=0x7f7f3806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:12.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f7f3806ca10 0x7f7f3806eed0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c001f10 tx=0x7f7f3c009040 comp rx=0 tx=0).stop 2024-04-26T19:52:12.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 msgr2=0x7f7f4c0b2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:12.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 0x7f7f4c0b2df0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7f4402a340 tx=0x7f7f440046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:12.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 shutdown_connections 2024-04-26T19:52:12.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f7f3806ca10 0x7f7f3806eed0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:12.221 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f7f4c0d2cd0 0x7f7f4c0b2df0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:12.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f7f4c06ff00 0x7f7f4c1f7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:12.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 --2- 172.21.15.79:0/2091377935 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f7f4c06fb20 0x7f7f4c0b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:12.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 >> 172.21.15.79:0/2091377935 conn(0x7f7f4c071c30 msgr2=0x7f7f4c06e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:12.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 shutdown_connections 2024-04-26T19:52:12.222 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:12.220+0000 7f7f517f7640 1 -- 172.21.15.79:0/2091377935 wait complete. 2024-04-26T19:52:12.232 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2024-04-26T19:52:12.233 INFO:tasks.cephfs_test_runner: 2024-04-26T19:52:12.233 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-26T19:52:12.233 INFO:tasks.cephfs_test_runner:Ran 1 test in 54.285s 2024-04-26T19:52:12.233 INFO:tasks.cephfs_test_runner: 2024-04-26T19:52:12.233 INFO:tasks.cephfs_test_runner:OK 2024-04-26T19:52:12.234 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-26T19:52:12.245 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T19:52:12.269 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T19:52:12.269 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:52:12.269 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:52:12.285 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056106,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17615 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T19:52:12.286 DEBUG:teuthology.orchestra.run.smithi079:> set -ex 2024-04-26T19:52:12.286 DEBUG:teuthology.orchestra.run.smithi079:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:52:12.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056106,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17615 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497688k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-26T19:52:12.341 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:52:12.341 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:52:12.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056107,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T19:52:12.360 DEBUG:teuthology.orchestra.run.smithi083:> set -ex 2024-04-26T19:52:12.360 DEBUG:teuthology.orchestra.run.smithi083:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T19:52:12.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056107,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=251 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-26T19:52:12.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.419 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-04-26T19:52:12.434 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-26T19:52:12.434 INFO:teuthology.orchestra.run.smithi079.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T19:52:12.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.435 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-26T19:52:12.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.499 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T19:52:12.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.564 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2024-04-26T19:52:12.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.659 DEBUG:teuthology.orchestra.run.smithi079:> ip netns list 2024-04-26T19:52:12.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.675 DEBUG:teuthology.orchestra.run.smithi079:> sudo ip link delete ceph-brx 2024-04-26T19:52:12.741 INFO:teuthology.orchestra.run.smithi079.stderr:Cannot find device "ceph-brx" 2024-04-26T19:52:12.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:52:12.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.743 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-26T19:52:12.761 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-26T19:52:12.761 INFO:teuthology.orchestra.run.smithi083.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T19:52:12.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.762 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-26T19:52:12.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.831 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T19:52:12.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:12.899 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-26T19:52:13.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:13.003 DEBUG:teuthology.orchestra.run.smithi083:> ip netns list 2024-04-26T19:52:13.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T19:52:13.018 DEBUG:teuthology.orchestra.run.smithi083:> sudo ip link delete ceph-brx 2024-04-26T19:52:13.086 INFO:teuthology.orchestra.run.smithi083.stderr:Cannot find device "ceph-brx" 2024-04-26T19:52:13.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:52:13.087 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T19:52:13.098 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T19:52:13.098 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:52:13.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.198+0000 7f6f5eb4e640 1 Processor -- start 2024-04-26T19:52:13.199 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.198+0000 7f6f5eb4e640 1 -- start start 2024-04-26T19:52:13.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f5eb4e640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6f580cd9c0 0x7f6f580b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f5eb4e640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f580ce1a0 con 0x7f6f580735a0 2024-04-26T19:52:13.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f5eb4e640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f580cdec0 con 0x7f6f58156670 2024-04-26T19:52:13.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f5eb4e640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f6f58073110 con 0x7f6f580cd9c0 2024-04-26T19:52:13.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f57fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f6f580cd9c0 0x7f6f580b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f5d0eb640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f6f580735a0 0x7f6f580cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50900/0 (socket says 172.21.15.79:50900) 2024-04-26T19:52:13.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f5d0eb640 1 -- 172.21.15.79:0/3230185134 learned_addr learned my addr 172.21.15.79:0/3230185134 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:13.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f57fff640 1 -- 172.21.15.79:0/3230185134 >> v1:172.21.15.83:6789/0 conn(0x7f6f58156670 legacy=0x7f6f5815d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.200 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f57fff640 1 -- 172.21.15.79:0/3230185134 >> v1:172.21.15.79:6789/0 conn(0x7f6f580735a0 legacy=0x7f6f580cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f57fff640 1 -- 172.21.15.79:0/3230185134 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f58072ac0 con 0x7f6f580cd9c0 2024-04-26T19:52:13.201 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.199+0000 7f6f577fe640 1 -- 172.21.15.79:0/3230185134 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 702843330 0 0) 0x7f6f580ce1a0 con 0x7f6f580735a0 2024-04-26T19:52:13.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.200+0000 7f6f57fff640 1 --2- 172.21.15.79:0/3230185134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f580cd9c0 0x7f6f580b3de0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6f440097a0 tx=0x7f6f4402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dfcce8768df8f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:13.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.200+0000 7f6f577fe640 1 -- 172.21.15.79:0/3230185134 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44037070 con 0x7f6f580cd9c0 2024-04-26T19:52:13.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.200+0000 7f6f577fe640 1 -- 172.21.15.79:0/3230185134 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f44032440 con 0x7f6f580cd9c0 2024-04-26T19:52:13.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.200+0000 7f6f577fe640 1 -- 172.21.15.79:0/3230185134 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f44032810 con 0x7f6f580cd9c0 2024-04-26T19:52:13.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.201+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3230185134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f580cd9c0 msgr2=0x7f6f580b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.201+0000 7f6f5eb4e640 1 --2- 172.21.15.79:0/3230185134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f580cd9c0 0x7f6f580b3de0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6f440097a0 tx=0x7f6f4402a230 comp rx=0 tx=0).stop 2024-04-26T19:52:13.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.201+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3230185134 shutdown_connections 2024-04-26T19:52:13.202 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.201+0000 7f6f5eb4e640 1 --2- 172.21.15.79:0/3230185134 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f580cd9c0 0x7f6f580b3de0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.201+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3230185134 >> 172.21.15.79:0/3230185134 conn(0x7f6f580b39d0 msgr2=0x7f6f5806feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:13.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.201+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3230185134 shutdown_connections 2024-04-26T19:52:13.203 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.201+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3230185134 wait complete. 2024-04-26T19:52:13.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.202+0000 7f6f5eb4e640 1 Processor -- start 2024-04-26T19:52:13.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.202+0000 7f6f5eb4e640 1 -- start start 2024-04-26T19:52:13.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5eb4e640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f580735a0 0x7f6f5814a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5eb4e640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6f580cd9c0 0x7f6f58152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5eb4e640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f58156670 0x7f6f580b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5eb4e640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f6f58156bd0 con 0x7f6f58156670 2024-04-26T19:52:13.205 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5eb4e640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f6f58156a50 con 0x7f6f580735a0 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5eb4e640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f6f580b4320 con 0x7f6f580cd9c0 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5d0eb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f58156670 0x7f6f580b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5d0eb640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f58156670 0x7f6f580b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44198/0 (socket says 172.21.15.79:44198) 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5d0eb640 1 -- 172.21.15.79:0/3417927999 learned_addr learned my addr 172.21.15.79:0/3417927999 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5c8ea640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f580735a0 0x7f6f5814a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5d0eb640 1 -- 172.21.15.79:0/3417927999 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6f580cd9c0 msgr2=0x7f6f58152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5d0eb640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6f580cd9c0 0x7f6f58152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5d0eb640 1 -- 172.21.15.79:0/3417927999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f580735a0 msgr2=0x7f6f5814a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5d0eb640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f580735a0 0x7f6f5814a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5d0eb640 1 -- 172.21.15.79:0/3417927999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f5814a5a0 con 0x7f6f58156670 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5c8ea640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f580735a0 0x7f6f5814a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.203+0000 7f6f5d0eb640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f58156670 0x7f6f580b1450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c7d0 tx=0x7f6f5000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:13.206 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.204+0000 7f6f557fa640 1 -- 172.21.15.79:0/3417927999 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f50018070 con 0x7f6f58156670 2024-04-26T19:52:13.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.204+0000 7f6f557fa640 1 -- 172.21.15.79:0/3417927999 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f5000a5e0 con 0x7f6f58156670 2024-04-26T19:52:13.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.204+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f581f5f50 con 0x7f6f58156670 2024-04-26T19:52:13.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.204+0000 7f6f557fa640 1 -- 172.21.15.79:0/3417927999 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f5000aa10 con 0x7f6f58156670 2024-04-26T19:52:13.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.204+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f58072060 con 0x7f6f58156670 2024-04-26T19:52:13.207 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.205+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f581f6120 con 0x7f6f58156670 2024-04-26T19:52:13.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.205+0000 7f6f557fa640 1 -- 172.21.15.79:0/3417927999 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f6f5001b070 con 0x7f6f58156670 2024-04-26T19:52:13.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.205+0000 7f6f557fa640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f6f3c06ca40 0x7f6f3c06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.206+0000 7f6f5c8ea640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f6f3c06ca40 0x7f6f3c06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.206+0000 7f6f557fa640 1 -- 172.21.15.79:0/3417927999 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f6f5008b3a0 con 0x7f6f58156670 2024-04-26T19:52:13.208 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.206+0000 7f6f5c8ea640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f6f3c06ca40 0x7f6f3c06ef00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c000f60 tx=0x7f6f4c009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:13.210 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.210+0000 7f6f557fa640 1 -- 172.21.15.79:0/3417927999 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f6f5814a5a0 con 0x7f6f58156670 2024-04-26T19:52:13.382 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.382+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f580b3f30 con 0x7f6f3c06ca40 2024-04-26T19:52:13.383 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.383+0000 7f6f557fa640 1 -- 172.21.15.79:0/3417927999 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f6f580b3f30 con 0x7f6f3c06ca40 2024-04-26T19:52:13.383 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:13.384 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:52:13.385 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f6f3c06ca40 msgr2=0x7f6f3c06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f6f3c06ca40 0x7f6f3c06ef00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c000f60 tx=0x7f6f4c009040 comp rx=0 tx=0).stop 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f58156670 msgr2=0x7f6f580b1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f58156670 0x7f6f580b1450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6f5000c7d0 tx=0x7f6f5000cc90 comp rx=0 tx=0).stop 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 shutdown_connections 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f6f3c06ca40 0x7f6f3c06ef00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f6f58156670 0x7f6f580b1450 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6f580cd9c0 0x7f6f58152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 --2- 172.21.15.79:0/3417927999 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6f580735a0 0x7f6f5814a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 >> 172.21.15.79:0/3417927999 conn(0x7f6f580b39d0 msgr2=0x7f6f580b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 shutdown_connections 2024-04-26T19:52:13.386 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.385+0000 7f6f5eb4e640 1 -- 172.21.15.79:0/3417927999 wait complete. 2024-04-26T19:52:13.396 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-26T19:52:13.214071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225852,"kb_used_data":5524,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748868,"statfs":{"total":773094113280,"available":772862840832,"internally_reserved":0,"allocated":5656576,"data_stored":3388552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-733985,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-321,"num_read_kb":-165,"num_write":-341,"num_write_kb":-2907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.068178"},"pg_stats":[{"pgid":"1.0","version":"42'139","reported_seq":292,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T19:52:10.859378+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:52:10.859378+0000","last_peered":"2024-04-26T19:52:10.859378+0000","last_clean":"2024-04-26T19:52:10.859378+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:52:10.859378+0000","last_undegraded":"2024-04-26T19:52:10.859378+0000","last_fullsized":"2024-04-26T19:52:10.859378+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":822220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":611624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":225293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":225293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":808698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:52:13.396 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:52:13.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.492+0000 7f93a6dc9640 1 Processor -- start 2024-04-26T19:52:13.493 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.493+0000 7f93a6dc9640 1 -- start start 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.493+0000 7f93a6dc9640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f93a006fb20 0x7f93a00d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.493+0000 7f93a6dc9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a006ff00 0x7f93a00d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.493+0000 7f93a6dc9640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93a00b1290 con 0x7f93a01488c0 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.493+0000 7f93a6dc9640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f93a00d3e60 con 0x7f93a006fb20 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.493+0000 7f93a6dc9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f93a00d4090 con 0x7f93a006ff00 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.493+0000 7f93a4b65640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f93a006fb20 0x7f93a00d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.493+0000 7f93a4b65640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f93a006fb20 0x7f93a00d2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44210/0 (socket says 172.21.15.79:44210) 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.494+0000 7f93a4b65640 1 -- 172.21.15.79:0/689933919 learned_addr learned my addr 172.21.15.79:0/689933919 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.494+0000 7f939ffff640 1 --2- 172.21.15.79:0/689933919 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a006ff00 0x7f93a00d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.494+0000 7f93a4b65640 1 -- 172.21.15.79:0/689933919 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a006ff00 msgr2=0x7f93a00d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.494+0000 7f93a4b65640 1 --2- 172.21.15.79:0/689933919 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a006ff00 0x7f93a00d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.494+0000 7f93a4b65640 1 -- 172.21.15.79:0/689933919 >> v1:172.21.15.83:6789/0 conn(0x7f93a01488c0 legacy=0x7f93a0148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.496 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.494+0000 7f93a4b65640 1 -- 172.21.15.79:0/689933919 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a00d4370 con 0x7f93a006fb20 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.494+0000 7f93a4b65640 1 --2- 172.21.15.79:0/689933919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 0x7f93a00d2cd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9394009870 tx=0x7f939402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81f60f787b41c38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f939f7fe640 1 -- 172.21.15.79:0/689933919 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394037070 con 0x7f93a006fb20 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f939f7fe640 1 -- 172.21.15.79:0/689933919 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939402ae50 con 0x7f93a006fb20 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f939f7fe640 1 -- 172.21.15.79:0/689933919 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394032860 con 0x7f93a006fb20 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f93a6dc9640 1 -- 172.21.15.79:0/689933919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 msgr2=0x7f93a00d2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f93a6dc9640 1 --2- 172.21.15.79:0/689933919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 0x7f93a00d2cd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f9394009870 tx=0x7f939402a300 comp rx=0 tx=0).stop 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f93a6dc9640 1 -- 172.21.15.79:0/689933919 shutdown_connections 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f93a6dc9640 1 --2- 172.21.15.79:0/689933919 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a006ff00 0x7f93a00d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f93a6dc9640 1 --2- 172.21.15.79:0/689933919 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 0x7f93a00d2cd0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f93a6dc9640 1 -- 172.21.15.79:0/689933919 >> 172.21.15.79:0/689933919 conn(0x7f93a0071c30 msgr2=0x7f93a006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f93a6dc9640 1 -- 172.21.15.79:0/689933919 shutdown_connections 2024-04-26T19:52:13.497 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.495+0000 7f93a6dc9640 1 -- 172.21.15.79:0/689933919 wait complete. 2024-04-26T19:52:13.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.496+0000 7f93a6dc9640 1 Processor -- start 2024-04-26T19:52:13.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.496+0000 7f93a6dc9640 1 -- start start 2024-04-26T19:52:13.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a6dc9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 0x7f93a01f9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a6dc9640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a006ff00 0x7f93a015e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a6dc9640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a01488c0 0x7f93a013cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.500 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a6dc9640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f93a00d4c10 con 0x7f93a006fb20 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a6dc9640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f93a00d4930 con 0x7f93a006ff00 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a6dc9640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f93a00aebb0 con 0x7f93a01488c0 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a4b65640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 0x7f93a01f9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a4b65640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 0x7f93a01f9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44212/0 (socket says 172.21.15.79:44212) 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a4b65640 1 -- 172.21.15.79:0/1343562463 learned_addr learned my addr 172.21.15.79:0/1343562463 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f939ffff640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a006ff00 0x7f93a015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a5366640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a01488c0 0x7f93a013cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a4b65640 1 -- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a01488c0 msgr2=0x7f93a013cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a4b65640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a01488c0 0x7f93a013cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a4b65640 1 -- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a006ff00 msgr2=0x7f93a015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a4b65640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a006ff00 0x7f93a015e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.501 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.497+0000 7f93a4b65640 1 -- 172.21.15.79:0/1343562463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a013d590 con 0x7f93a006fb20 2024-04-26T19:52:13.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.498+0000 7f939ffff640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a006ff00 0x7f93a015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:13.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.498+0000 7f93a5366640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a01488c0 0x7f93a013cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:13.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.498+0000 7f93a4b65640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 0x7f93a01f9510 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f939402a340 tx=0x7f93940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:13.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.498+0000 7f939d7fa640 1 -- 172.21.15.79:0/1343562463 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394041020 con 0x7f93a006fb20 2024-04-26T19:52:13.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.498+0000 7f939d7fa640 1 -- 172.21.15.79:0/1343562463 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939403c670 con 0x7f93a006fb20 2024-04-26T19:52:13.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.498+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a013d870 con 0x7f93a006fb20 2024-04-26T19:52:13.502 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.498+0000 7f939d7fa640 1 -- 172.21.15.79:0/1343562463 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394037040 con 0x7f93a006fb20 2024-04-26T19:52:13.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.498+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a01f8d10 con 0x7f93a006fb20 2024-04-26T19:52:13.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.499+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93a00506c0 con 0x7f93a006fb20 2024-04-26T19:52:13.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.499+0000 7f939d7fa640 1 -- 172.21.15.79:0/1343562463 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f9394044020 con 0x7f93a006fb20 2024-04-26T19:52:13.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.500+0000 7f939d7fa640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f938406ca40 0x7f938406ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.500+0000 7f939d7fa640 1 -- 172.21.15.79:0/1343562463 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f93940ac0a0 con 0x7f93a006fb20 2024-04-26T19:52:13.503 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.500+0000 7f939ffff640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f938406ca40 0x7f938406ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.500+0000 7f939ffff640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f938406ca40 0x7f938406ef00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93a0141290 tx=0x7f938c008040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:13.504 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.504+0000 7f939d7fa640 1 -- 172.21.15.79:0/1343562463 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f939404b070 con 0x7f93a006fb20 2024-04-26T19:52:13.676 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.676+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93a00d3700 con 0x7f938406ca40 2024-04-26T19:52:13.677 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.676+0000 7f939d7fa640 1 -- 172.21.15.79:0/1343562463 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f93a00d3700 con 0x7f938406ca40 2024-04-26T19:52:13.677 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:13.678 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:52:13.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f938406ca40 msgr2=0x7f938406ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.679 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f938406ca40 0x7f938406ef00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f93a0141290 tx=0x7f938c008040 comp rx=0 tx=0).stop 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 msgr2=0x7f93a01f9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 0x7f93a01f9510 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f939402a340 tx=0x7f93940046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 shutdown_connections 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f938406ca40 0x7f938406ef00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f93a01488c0 0x7f93a013cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f93a006ff00 0x7f93a015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 --2- 172.21.15.79:0/1343562463 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f93a006fb20 0x7f93a01f9510 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 >> 172.21.15.79:0/1343562463 conn(0x7f93a0071c30 msgr2=0x7f93a006e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 shutdown_connections 2024-04-26T19:52:13.680 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.678+0000 7f93a6dc9640 1 -- 172.21.15.79:0/1343562463 wait complete. 2024-04-26T19:52:13.690 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-26T19:52:13.214071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225852,"kb_used_data":5524,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748868,"statfs":{"total":773094113280,"available":772862840832,"internally_reserved":0,"allocated":5656576,"data_stored":3388552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-733985,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-321,"num_read_kb":-165,"num_write":-341,"num_write_kb":-2907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.068178"},"pg_stats":[{"pgid":"1.0","version":"42'139","reported_seq":292,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T19:52:10.859378+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:52:10.859378+0000","last_peered":"2024-04-26T19:52:10.859378+0000","last_clean":"2024-04-26T19:52:10.859378+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:52:10.859378+0000","last_undegraded":"2024-04-26T19:52:10.859378+0000","last_fullsized":"2024-04-26T19:52:10.859378+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":822220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":611624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":225293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":225293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":808698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:52:13.690 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T19:52:13.690 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:52:13.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.787+0000 7fef0aedc640 1 Processor -- start 2024-04-26T19:52:13.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.787+0000 7fef0aedc640 1 -- start start 2024-04-26T19:52:13.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef0aedc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fef0406ce70 0x7fef0406d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef0aedc640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fef04140a00 0x7fef04140e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef0aedc640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef0415d7c0 con 0x7fef040b4c90 2024-04-26T19:52:13.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef0aedc640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fef0415da80 con 0x7fef0406ce70 2024-04-26T19:52:13.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef0aedc640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fef0415dd60 con 0x7fef04140a00 2024-04-26T19:52:13.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef03fff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fef0406ce70 0x7fef0406d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef08c78640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fef040b4c90 0x7fef040c8810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50914/0 (socket says 172.21.15.79:50914) 2024-04-26T19:52:13.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef08c78640 1 -- 172.21.15.79:0/3171351594 learned_addr learned my addr 172.21.15.79:0/3171351594 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:13.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef09479640 1 --2- 172.21.15.79:0/3171351594 >> v2:172.21.15.83:3300/0 conn(0x7fef04140a00 0x7fef04140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef03fff640 1 -- 172.21.15.79:0/3171351594 >> v2:172.21.15.83:3300/0 conn(0x7fef04140a00 msgr2=0x7fef04140e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:13.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef03fff640 1 --2- 172.21.15.79:0/3171351594 >> v2:172.21.15.83:3300/0 conn(0x7fef04140a00 0x7fef04140e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef03fff640 1 -- 172.21.15.79:0/3171351594 >> v1:172.21.15.79:6789/0 conn(0x7fef040b4c90 legacy=0x7fef040c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef037fe640 1 -- 172.21.15.79:0/3171351594 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2393812389 0 0) 0x7fef0415d7c0 con 0x7fef040b4c90 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.788+0000 7fef03fff640 1 -- 172.21.15.79:0/3171351594 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef0415e040 con 0x7fef0406ce70 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef03fff640 1 --2- 172.21.15.79:0/3171351594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef0406ce70 0x7fef0406d320 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fef040c5360 tx=0x7feef002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b8dca8365b701f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef037fe640 1 -- 172.21.15.79:0/3171351594 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0037070 con 0x7fef0406ce70 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef037fe640 1 -- 172.21.15.79:0/3171351594 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef0032440 con 0x7fef0406ce70 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef037fe640 1 -- 172.21.15.79:0/3171351594 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef002ae60 con 0x7fef0406ce70 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef0aedc640 1 -- 172.21.15.79:0/3171351594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef0406ce70 msgr2=0x7fef0406d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef0aedc640 1 --2- 172.21.15.79:0/3171351594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef0406ce70 0x7fef0406d320 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fef040c5360 tx=0x7feef002a110 comp rx=0 tx=0).stop 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef0aedc640 1 -- 172.21.15.79:0/3171351594 shutdown_connections 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef0aedc640 1 --2- 172.21.15.79:0/3171351594 >> v2:172.21.15.83:3300/0 conn(0x7fef04140a00 0x7fef04140e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef0aedc640 1 --2- 172.21.15.79:0/3171351594 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef0406ce70 0x7fef0406d320 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef0aedc640 1 -- 172.21.15.79:0/3171351594 >> 172.21.15.79:0/3171351594 conn(0x7fef0406c350 msgr2=0x7fef0406c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef0aedc640 1 -- 172.21.15.79:0/3171351594 shutdown_connections 2024-04-26T19:52:13.792 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.789+0000 7fef0aedc640 1 -- 172.21.15.79:0/3171351594 wait complete. 2024-04-26T19:52:13.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.791+0000 7fef0aedc640 1 Processor -- start 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.791+0000 7fef0aedc640 1 -- start start 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.791+0000 7fef0aedc640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef0406ce70 0x7fef0413dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.791+0000 7fef0aedc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef040b4c90 0x7fef040cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.791+0000 7fef0aedc640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef04140a00 0x7fef0406f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.791+0000 7fef0aedc640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fef040c83a0 con 0x7fef040b4c90 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.791+0000 7fef0aedc640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fef040b5070 con 0x7fef04140a00 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.791+0000 7fef0aedc640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fef040c8520 con 0x7fef0406ce70 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef03fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef040b4c90 0x7fef040cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef03fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef040b4c90 0x7fef040cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44234/0 (socket says 172.21.15.79:44234) 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef03fff640 1 -- 172.21.15.79:0/4055344743 learned_addr learned my addr 172.21.15.79:0/4055344743 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:13.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef08c78640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef0406ce70 0x7fef0413dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef09479640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef04140a00 0x7fef0406f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef03fff640 1 -- 172.21.15.79:0/4055344743 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef0406ce70 msgr2=0x7fef0413dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef03fff640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef0406ce70 0x7fef0413dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef03fff640 1 -- 172.21.15.79:0/4055344743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef04140a00 msgr2=0x7fef0406f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef03fff640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef04140a00 0x7fef0406f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef03fff640 1 -- 172.21.15.79:0/4055344743 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef041f61a0 con 0x7fef040b4c90 2024-04-26T19:52:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef03fff640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef040b4c90 0x7fef040cf310 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feef0000c00 tx=0x7feef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:13.797 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef017fa640 1 -- 172.21.15.79:0/4055344743 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0043070 con 0x7fef040b4c90 2024-04-26T19:52:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef017fa640 1 -- 172.21.15.79:0/4055344743 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef0038040 con 0x7fef040b4c90 2024-04-26T19:52:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef041f6480 con 0x7fef040b4c90 2024-04-26T19:52:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef017fa640 1 -- 172.21.15.79:0/4055344743 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef0037050 con 0x7fef040b4c90 2024-04-26T19:52:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.792+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef041f6a40 con 0x7fef040b4c90 2024-04-26T19:52:13.799 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.793+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef040506c0 con 0x7fef040b4c90 2024-04-26T19:52:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.793+0000 7fef017fa640 1 -- 172.21.15.79:0/4055344743 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7feef0046070 con 0x7fef040b4c90 2024-04-26T19:52:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.794+0000 7fef017fa640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feee806ca10 0x7feee806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.794+0000 7fef017fa640 1 -- 172.21.15.79:0/4055344743 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7feef00abe90 con 0x7fef040b4c90 2024-04-26T19:52:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.794+0000 7fef08c78640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feee806ca10 0x7feee806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:13.800 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.794+0000 7fef08c78640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feee806ca10 0x7feee806eed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feef8007fd0 tx=0x7feef8007de0 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:13.801 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.798+0000 7fef017fa640 1 -- 172.21.15.79:0/4055344743 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7feef0079b00 con 0x7fef040b4c90 2024-04-26T19:52:13.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.969+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef041f5f50 con 0x7feee806ca10 2024-04-26T19:52:13.971 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.970+0000 7fef017fa640 1 -- 172.21.15.79:0/4055344743 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7fef041f5f50 con 0x7feee806ca10 2024-04-26T19:52:13.971 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:13.971 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:52:13.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feee806ca10 msgr2=0x7feee806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feee806ca10 0x7feee806eed0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7feef8007fd0 tx=0x7feef8007de0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef040b4c90 msgr2=0x7fef040cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:13.973 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef040b4c90 0x7fef040cf310 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feef0000c00 tx=0x7feef00046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 shutdown_connections 2024-04-26T19:52:13.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7feee806ca10 0x7feee806eed0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fef04140a00 0x7fef0406f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fef040b4c90 0x7fef040cf310 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 --2- 172.21.15.79:0/4055344743 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fef0406ce70 0x7fef0413dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:13.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 >> 172.21.15.79:0/4055344743 conn(0x7fef0406c350 msgr2=0x7fef0406c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:13.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 shutdown_connections 2024-04-26T19:52:13.974 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:13.972+0000 7fef0aedc640 1 -- 172.21.15.79:0/4055344743 wait complete. 2024-04-26T19:52:13.984 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-26T19:52:13.214071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225852,"kb_used_data":5524,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748868,"statfs":{"total":773094113280,"available":772862840832,"internally_reserved":0,"allocated":5656576,"data_stored":3388552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-733985,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-321,"num_read_kb":-165,"num_write":-341,"num_write_kb":-2907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.068178"},"pg_stats":[{"pgid":"1.0","version":"42'139","reported_seq":292,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T19:52:10.859378+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:52:10.859378+0000","last_peered":"2024-04-26T19:52:10.859378+0000","last_clean":"2024-04-26T19:52:10.859378+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:52:10.859378+0000","last_undegraded":"2024-04-26T19:52:10.859378+0000","last_fullsized":"2024-04-26T19:52:10.859378+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":822220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":611624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":503808,"data_stored":225293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":225293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":808698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:52:13.985 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T19:52:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.080+0000 7f5253c86640 1 Processor -- start 2024-04-26T19:52:14.081 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.080+0000 7f5253c86640 1 -- start start 2024-04-26T19:52:14.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5253c86640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d0e0 0x7f524c163aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:14.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5253c86640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f524c06d4c0 0x7f524c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:14.083 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5253c86640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f524c0b4fc0 con 0x7f524c1487c0 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5253c86640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f524c0b4d60 con 0x7f524c06d4c0 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5253c86640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f524c0c9130 con 0x7f524c06d0e0 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5252223640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f524c1487c0 0x7f524c0720e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50918/0 (socket says 172.21.15.79:50918) 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5252223640 1 -- 172.21.15.79:0/4057478897 learned_addr learned my addr 172.21.15.79:0/4057478897 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5251a22640 1 --2- 172.21.15.79:0/4057478897 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d0e0 0x7f524c163aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5251221640 1 --2- 172.21.15.79:0/4057478897 >> v2:172.21.15.83:3300/0 conn(0x7f524c06d4c0 0x7f524c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5250a20640 1 -- 172.21.15.79:0/4057478897 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 413606612 0 0) 0x7f524c0b4fc0 con 0x7f524c1487c0 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.081+0000 7f5250a20640 1 -- 172.21.15.79:0/4057478897 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5238003520 con 0x7f524c1487c0 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.082+0000 7f5251a22640 1 -- 172.21.15.79:0/4057478897 >> v2:172.21.15.83:3300/0 conn(0x7f524c06d4c0 msgr2=0x7f524c0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.082+0000 7f5251a22640 1 --2- 172.21.15.79:0/4057478897 >> v2:172.21.15.83:3300/0 conn(0x7f524c06d4c0 0x7f524c0b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.082+0000 7f5250a20640 1 -- 172.21.15.79:0/4057478897 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1573842847 0 0) 0x7f5238003520 con 0x7f524c1487c0 2024-04-26T19:52:14.084 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.082+0000 7f5251a22640 1 -- 172.21.15.79:0/4057478897 >> v1:172.21.15.79:6789/0 conn(0x7f524c1487c0 legacy=0x7f524c0720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:14.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.082+0000 7f5251a22640 1 -- 172.21.15.79:0/4057478897 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c0c9410 con 0x7f524c06d0e0 2024-04-26T19:52:14.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.082+0000 7f5251221640 1 --2- 172.21.15.79:0/4057478897 >> v2:172.21.15.83:3300/0 conn(0x7f524c06d4c0 0x7f524c0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:14.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.082+0000 7f5251a22640 1 --2- 172.21.15.79:0/4057478897 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d0e0 0x7f524c163aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5248009d90 tx=0x7f524802a560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b60488fe6f6c9cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:14.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5250a20640 1 -- 172.21.15.79:0/4057478897 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5244001670 con 0x7f524c06d0e0 2024-04-26T19:52:14.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5250a20640 1 -- 172.21.15.79:0/4057478897 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5248007540 con 0x7f524c06d0e0 2024-04-26T19:52:14.085 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5250a20640 1 -- 172.21.15.79:0/4057478897 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52480078f0 con 0x7f524c06d0e0 2024-04-26T19:52:14.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5253c86640 1 -- 172.21.15.79:0/4057478897 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d0e0 msgr2=0x7f524c163aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:14.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5253c86640 1 --2- 172.21.15.79:0/4057478897 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d0e0 0x7f524c163aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5248009d90 tx=0x7f524802a560 comp rx=0 tx=0).stop 2024-04-26T19:52:14.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5253c86640 1 -- 172.21.15.79:0/4057478897 shutdown_connections 2024-04-26T19:52:14.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5253c86640 1 --2- 172.21.15.79:0/4057478897 >> v2:172.21.15.83:3300/0 conn(0x7f524c06d4c0 0x7f524c0b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:14.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5253c86640 1 --2- 172.21.15.79:0/4057478897 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d0e0 0x7f524c163aa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:14.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5253c86640 1 -- 172.21.15.79:0/4057478897 >> 172.21.15.79:0/4057478897 conn(0x7f524c06c350 msgr2=0x7f524c06c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:14.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5253c86640 1 -- 172.21.15.79:0/4057478897 shutdown_connections 2024-04-26T19:52:14.088 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.083+0000 7f5253c86640 1 -- 172.21.15.79:0/4057478897 wait complete. 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.084+0000 7f5253c86640 1 Processor -- start 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.084+0000 7f5253c86640 1 -- start start 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5253c86640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f524c06d0e0 0x7f524c1512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5253c86640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d4c0 0x7f524c0b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5253c86640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f524c1487c0 0x7f524c152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5253c86640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f524c0caf00 con 0x7f524c06d0e0 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5253c86640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f524c0c99d0 con 0x7f524c1487c0 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5253c86640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f524c0cc510 con 0x7f524c06d4c0 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5251a22640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f524c06d0e0 0x7f524c1512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5251a22640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f524c06d0e0 0x7f524c1512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44248/0 (socket says 172.21.15.79:44248) 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5251a22640 1 -- 172.21.15.79:0/2608255882 learned_addr learned my addr 172.21.15.79:0/2608255882 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5251221640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d4c0 0x7f524c0b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5252223640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f524c1487c0 0x7f524c152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:14.089 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5251a22640 1 -- 172.21.15.79:0/2608255882 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d4c0 msgr2=0x7f524c0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5251a22640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d4c0 0x7f524c0b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5251a22640 1 -- 172.21.15.79:0/2608255882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f524c1487c0 msgr2=0x7f524c152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5251a22640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f524c1487c0 0x7f524c152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.085+0000 7f5251a22640 1 -- 172.21.15.79:0/2608255882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c1519e0 con 0x7f524c06d0e0 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.086+0000 7f5251a22640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f524c06d0e0 0x7f524c1512e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f524802ab50 tx=0x7f5248004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.086+0000 7f52427fc640 1 -- 172.21.15.79:0/2608255882 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248009450 con 0x7f524c06d0e0 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.086+0000 7f52427fc640 1 -- 172.21.15.79:0/2608255882 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5248004240 con 0x7f524c06d0e0 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.086+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c151cc0 con 0x7f524c06d0e0 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.086+0000 7f52427fc640 1 -- 172.21.15.79:0/2608255882 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52480045f0 con 0x7f524c06d0e0 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.086+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c152280 con 0x7f524c06d0e0 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.087+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524c0506c0 con 0x7f524c06d0e0 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.087+0000 7f52427fc640 1 -- 172.21.15.79:0/2608255882 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f524803b410 con 0x7f524c06d0e0 2024-04-26T19:52:14.090 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.088+0000 7f52427fc640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f523006ca10 0x7f523006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:14.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.088+0000 7f52427fc640 1 -- 172.21.15.79:0/2608255882 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f52480ad3d0 con 0x7f524c06d0e0 2024-04-26T19:52:14.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.088+0000 7f5251221640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f523006ca10 0x7f523006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:14.091 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.088+0000 7f5251221640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f523006ca10 0x7f523006eed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f523c009500 tx=0x7f523c009110 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:14.092 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.092+0000 7f52427fc640 1 -- 172.21.15.79:0/2608255882 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f524803f030 con 0x7f524c06d0e0 2024-04-26T19:52:14.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.270+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f524c0d0d40 con 0x7f524c06d0e0 2024-04-26T19:52:14.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.271+0000 7f52427fc640 1 -- 172.21.15.79:0/2608255882 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+14122 (secure 0 0 0) 0x7f524807ae10 con 0x7f524c06d0e0 2024-04-26T19:52:14.272 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:14.272 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":44,"fsid":"b372f41f-f07b-4764-9bef-bcff62f9a5d2","created":"2024-04-26T19:50:06.217279+0000","modified":"2024-04-26T19:52:10.208831+0000","last_up_change":"2024-04-26T19:50:17.165704+0000","last_in_change":"2024-04-26T19:50:08.434954+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T19:50:18.265547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"58c5c8de-2274-4989-bf9c-1fe2258a4928","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6800","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6801","nonce":2567076951}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6802","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6803","nonce":2567076951}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6806","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6807","nonce":2567076951}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6804","nonce":2567076951},{"type":"v1","addr":"172.21.15.79:6805","nonce":2567076951}]},"public_addr":"172.21.15.79:6801/2567076951","cluster_addr":"172.21.15.79:6803/2567076951","heartbeat_back_addr":"172.21.15.79:6807/2567076951","heartbeat_front_addr":"172.21.15.79:6805/2567076951","state":["exists","up"]},{"osd":1,"uuid":"e01f7537-cc44-4ed8-9ba3-8f61de55aac1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6824","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6825","nonce":2915211239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6826","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6827","nonce":2915211239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6830","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6831","nonce":2915211239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6828","nonce":2915211239},{"type":"v1","addr":"172.21.15.79:6829","nonce":2915211239}]},"public_addr":"172.21.15.79:6825/2915211239","cluster_addr":"172.21.15.79:6827/2915211239","heartbeat_back_addr":"172.21.15.79:6831/2915211239","heartbeat_front_addr":"172.21.15.79:6829/2915211239","state":["exists","up"]},{"osd":2,"uuid":"428e11d4-edb8-4003-a6b6-59141d177026","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6808","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6809","nonce":569648363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6810","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6811","nonce":569648363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6814","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6815","nonce":569648363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6812","nonce":569648363},{"type":"v1","addr":"172.21.15.79:6813","nonce":569648363}]},"public_addr":"172.21.15.79:6809/569648363","cluster_addr":"172.21.15.79:6811/569648363","heartbeat_back_addr":"172.21.15.79:6815/569648363","heartbeat_front_addr":"172.21.15.79:6813/569648363","state":["exists","up"]},{"osd":3,"uuid":"89c30ecc-bd6d-43a7-829e-57e01ad3179e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6816","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6817","nonce":1557374359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6818","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6819","nonce":1557374359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6822","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6823","nonce":1557374359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6820","nonce":1557374359},{"type":"v1","addr":"172.21.15.79:6821","nonce":1557374359}]},"public_addr":"172.21.15.79:6817/1557374359","cluster_addr":"172.21.15.79:6819/1557374359","heartbeat_back_addr":"172.21.15.79:6823/1557374359","heartbeat_front_addr":"172.21.15.79:6821/1557374359","state":["exists","up"]},{"osd":4,"uuid":"a88b8475-38de-4d46-bf92-12cb08f29cd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6808","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6809","nonce":3328215207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6810","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6811","nonce":3328215207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6814","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6815","nonce":3328215207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6812","nonce":3328215207},{"type":"v1","addr":"172.21.15.83:6813","nonce":3328215207}]},"public_addr":"172.21.15.83:6809/3328215207","cluster_addr":"172.21.15.83:6811/3328215207","heartbeat_back_addr":"172.21.15.83:6815/3328215207","heartbeat_front_addr":"172.21.15.83:6813/3328215207","state":["exists","up"]},{"osd":5,"uuid":"0334aebf-d29f-4622-bd30-c69630e9c7ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6824","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6825","nonce":3851137838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6826","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6827","nonce":3851137838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6830","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6831","nonce":3851137838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6828","nonce":3851137838},{"type":"v1","addr":"172.21.15.83:6829","nonce":3851137838}]},"public_addr":"172.21.15.83:6825/3851137838","cluster_addr":"172.21.15.83:6827/3851137838","heartbeat_back_addr":"172.21.15.83:6831/3851137838","heartbeat_front_addr":"172.21.15.83:6829/3851137838","state":["exists","up"]},{"osd":6,"uuid":"baf0c717-e2cd-4c39-aea1-5b69fab094f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6800","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6801","nonce":2962195076}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6802","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6803","nonce":2962195076}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6806","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6807","nonce":2962195076}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6804","nonce":2962195076},{"type":"v1","addr":"172.21.15.83:6805","nonce":2962195076}]},"public_addr":"172.21.15.83:6801/2962195076","cluster_addr":"172.21.15.83:6803/2962195076","heartbeat_back_addr":"172.21.15.83:6807/2962195076","heartbeat_front_addr":"172.21.15.83:6805/2962195076","state":["exists","up"]},{"osd":7,"uuid":"65254695-e7da-47cc-90a7-8a2f9ca0ce36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6816","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6817","nonce":4292059702}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6818","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6819","nonce":4292059702}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6822","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6823","nonce":4292059702}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6820","nonce":4292059702},{"type":"v1","addr":"172.21.15.83:6821","nonce":4292059702}]},"public_addr":"172.21.15.83:6817/4292059702","cluster_addr":"172.21.15.83:6819/4292059702","heartbeat_back_addr":"172.21.15.83:6823/4292059702","heartbeat_front_addr":"172.21.15.83:6821/4292059702","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.376174+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.599331+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.487896+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.647848+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.664740+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.617461+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.631236+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T19:50:13.567370+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.79:6835/2334214524":"2024-04-27T19:52:07.143786+0000","172.21.15.79:0/2026733823":"2024-04-27T19:52:05.198362+0000","172.21.15.79:6832/3161939478":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1948040587":"2024-04-27T19:51:58.575815+0000","172.21.15.79:6837/2334214524":"2024-04-27T19:52:07.143786+0000","172.21.15.79:0/3742475428":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1992052263":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/3466651560":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/1084811462":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/16732380":"2024-04-27T19:51:58.575815+0000","172.21.15.79:6832/3922824612":"2024-04-27T19:52:05.198362+0000","172.21.15.79:6833/3161939478":"2024-04-27T19:51:58.575815+0000","172.21.15.79:0/1790989087":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/2471022702":"2024-04-27T19:52:05.198362+0000","172.21.15.79:6833/3922824612":"2024-04-27T19:52:05.198362+0000","172.21.15.79:0/466097619":"2024-04-27T19:52:05.198362+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T19:52:14.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f523006ca10 msgr2=0x7f523006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:14.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f523006ca10 0x7f523006eed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f523c009500 tx=0x7f523c009110 comp rx=0 tx=0).stop 2024-04-26T19:52:14.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f524c06d0e0 msgr2=0x7f524c1512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:14.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f524c06d0e0 0x7f524c1512e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f524802ab50 tx=0x7f5248004a40 comp rx=0 tx=0).stop 2024-04-26T19:52:14.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 shutdown_connections 2024-04-26T19:52:14.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f523006ca10 0x7f523006eed0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:14.274 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f524c1487c0 0x7f524c152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:14.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f524c06d4c0 0x7f524c0b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:14.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 --2- 172.21.15.79:0/2608255882 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f524c06d0e0 0x7f524c1512e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:14.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 >> 172.21.15.79:0/2608255882 conn(0x7f524c06c350 msgr2=0x7f524c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:14.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 shutdown_connections 2024-04-26T19:52:14.275 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:14.273+0000 7f5253c86640 1 -- 172.21.15.79:0/2608255882 wait complete. 2024-04-26T19:52:15.286 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T19:52:15.286 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T19:52:15.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.398+0000 7f1a2e371640 1 Processor -- start 2024-04-26T19:52:15.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.398+0000 7f1a2e371640 1 -- start start 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.399+0000 7f1a2e371640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a2815e0e0 0x7f1a28143240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.399+0000 7f1a2e371640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a280b3740 0x7f1a280b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.399+0000 7f1a2e371640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a280aa8c0 con 0x7f1a28160ce0 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.399+0000 7f1a2e371640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f1a28140520 con 0x7f1a280b3740 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.399+0000 7f1a2e371640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1a280ab940 con 0x7f1a2815e0e0 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a277fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a280b3740 0x7f1a280b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a27fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a2815e0e0 0x7f1a28143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a277fe640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f1a280b3740 0x7f1a280b3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3300/0 says I am v2:172.21.15.79:37840/0 (socket says 172.21.15.79:37840) 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a2c90e640 1 --1- >> v1:172.21.15.83:6789/0 conn(0x7f1a28160ce0 0x7f1a28161170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.83:6789/0 says I am v1:172.21.15.79:45552/0 (socket says 172.21.15.79:45552) 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a277fe640 1 -- 172.21.15.79:0/54734179 learned_addr learned my addr 172.21.15.79:0/54734179 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a27fff640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a2815e0e0 0x7f1a28143240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.83:3301/0 says I am v2:172.21.15.79:38308/0 (socket says 172.21.15.79:38308) 2024-04-26T19:52:15.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a27fff640 1 -- 172.21.15.79:0/54734179 >> v2:172.21.15.83:3300/0 conn(0x7f1a280b3740 msgr2=0x7f1a280b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:15.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a27fff640 1 --2- 172.21.15.79:0/54734179 >> v2:172.21.15.83:3300/0 conn(0x7f1a280b3740 0x7f1a280b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a27fff640 1 -- 172.21.15.79:0/54734179 >> v1:172.21.15.83:6789/0 conn(0x7f1a28160ce0 legacy=0x7f1a28161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a27fff640 1 -- 172.21.15.79:0/54734179 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a280aaba0 con 0x7f1a2815e0e0 2024-04-26T19:52:15.402 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.400+0000 7f1a277fe640 1 --2- 172.21.15.79:0/54734179 >> v2:172.21.15.83:3300/0 conn(0x7f1a280b3740 0x7f1a280b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:15.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.401+0000 7f1a27fff640 1 --2- 172.21.15.79:0/54734179 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a2815e0e0 0x7f1a28143240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a180097a0 tx=0x7f1a1802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b8ceda0545798df server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.401+0000 7f1a26ffd640 1 -- 172.21.15.79:0/54734179 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18037070 con 0x7f1a2815e0e0 2024-04-26T19:52:15.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.401+0000 7f1a26ffd640 1 -- 172.21.15.79:0/54734179 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a18032440 con 0x7f1a2815e0e0 2024-04-26T19:52:15.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.401+0000 7f1a26ffd640 1 -- 172.21.15.79:0/54734179 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18032810 con 0x7f1a2815e0e0 2024-04-26T19:52:15.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.402+0000 7f1a2e371640 1 -- 172.21.15.79:0/54734179 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a2815e0e0 msgr2=0x7f1a28143240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.402+0000 7f1a2e371640 1 --2- 172.21.15.79:0/54734179 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a2815e0e0 0x7f1a28143240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1a180097a0 tx=0x7f1a1802a180 comp rx=0 tx=0).stop 2024-04-26T19:52:15.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.402+0000 7f1a2e371640 1 -- 172.21.15.79:0/54734179 shutdown_connections 2024-04-26T19:52:15.403 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.402+0000 7f1a2e371640 1 --2- 172.21.15.79:0/54734179 >> v2:172.21.15.83:3300/0 conn(0x7f1a280b3740 0x7f1a280b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.402+0000 7f1a2e371640 1 --2- 172.21.15.79:0/54734179 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a2815e0e0 0x7f1a28143240 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.402+0000 7f1a2e371640 1 -- 172.21.15.79:0/54734179 >> 172.21.15.79:0/54734179 conn(0x7f1a28068390 msgr2=0x7f1a280687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:15.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.402+0000 7f1a2e371640 1 -- 172.21.15.79:0/54734179 shutdown_connections 2024-04-26T19:52:15.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.402+0000 7f1a2e371640 1 -- 172.21.15.79:0/54734179 wait complete. 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.403+0000 7f1a2e371640 1 Processor -- start 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.403+0000 7f1a2e371640 1 -- start start 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a2e371640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a280b3740 0x7f1a28156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a2e371640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2815e0e0 0x7f1a2813e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a2e371640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a28160ce0 0x7f1a281f4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a2e371640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1a280b40c0 con 0x7f1a280b3740 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a2e371640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1a28143780 con 0x7f1a2815e0e0 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a2e371640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1a280ab160 con 0x7f1a28160ce0 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a27fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a280b3740 0x7f1a28156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a27fff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a280b3740 0x7f1a28156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44254/0 (socket says 172.21.15.79:44254) 2024-04-26T19:52:15.408 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a27fff640 1 -- 172.21.15.79:0/3584447358 learned_addr learned my addr 172.21.15.79:0/3584447358 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a277fe640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2815e0e0 0x7f1a2813e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a2c90e640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a28160ce0 0x7f1a281f4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a27fff640 1 -- 172.21.15.79:0/3584447358 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a28160ce0 msgr2=0x7f1a281f4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a27fff640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a28160ce0 0x7f1a281f4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a27fff640 1 -- 172.21.15.79:0/3584447358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2815e0e0 msgr2=0x7f1a2813e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a27fff640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1a2815e0e0 0x7f1a2813e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a27fff640 1 -- 172.21.15.79:0/3584447358 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a2813e5b0 con 0x7f1a280b3740 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a2c90e640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1a28160ce0 0x7f1a281f4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.404+0000 7f1a27fff640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a280b3740 0x7f1a28156920 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1a1802a1c0 tx=0x7f1a180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.405+0000 7f1a24ff9640 1 -- 172.21.15.79:0/3584447358 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18041020 con 0x7f1a280b3740 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.405+0000 7f1a24ff9640 1 -- 172.21.15.79:0/3584447358 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a18030700 con 0x7f1a280b3740 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.405+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a2806e460 con 0x7f1a280b3740 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.405+0000 7f1a24ff9640 1 -- 172.21.15.79:0/3584447358 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18037040 con 0x7f1a280b3740 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.405+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a2806ea20 con 0x7f1a280b3740 2024-04-26T19:52:15.409 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.405+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1a280506c0 con 0x7f1a280b3740 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.406+0000 7f1a24ff9640 1 -- 172.21.15.79:0/3584447358 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f1a18044020 con 0x7f1a280b3740 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.407+0000 7f1a24ff9640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1a1406ca10 0x7f1a1406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.407+0000 7f1a24ff9640 1 -- 172.21.15.79:0/3584447358 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f1a180acac0 con 0x7f1a280b3740 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.407+0000 7f1a277fe640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1a1406ca10 0x7f1a1406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.407+0000 7f1a24ff9640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7f1a14073400 0x7f1a14075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.407+0000 7f1a24ff9640 1 -- 172.21.15.79:0/3584447358 --> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1a18036e90 con 0x7f1a14073400 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.407+0000 7f1a24ff9640 1 -- 172.21.15.79:0/3584447358 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f1a180acf50 con 0x7f1a280b3740 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.407+0000 7f1a2c90e640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7f1a14073400 0x7f1a14075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.407+0000 7f1a277fe640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1a1406ca10 0x7f1a1406eed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009630 tx=0x7f1a1c009240 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.410 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.407+0000 7f1a2c90e640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7f1a14073400 0x7f1a14075840 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.411 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.408+0000 7f1a24ff9640 1 -- 172.21.15.79:0/3584447358 <== osd.0 v2:172.21.15.79:6800/2567076951 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1a18036e90 con 0x7f1a14073400 2024-04-26T19:52:15.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.421+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 --> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1a280d1400 con 0x7f1a14073400 2024-04-26T19:52:15.422 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.421+0000 7f1a24ff9640 1 -- 172.21.15.79:0/3584447358 <== osd.0 v2:172.21.15.79:6800/2567076951 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f1a280d1400 con 0x7f1a14073400 2024-04-26T19:52:15.422 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:52:15.422 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T19:52:15.422 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:52:15.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.421+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7f1a14073400 msgr2=0x7f1a14075840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.421+0000 7f1a2e371640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6800/2567076951,v1:172.21.15.79:6801/2567076951] conn(0x7f1a14073400 0x7f1a14075840 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.422+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1a1406ca10 msgr2=0x7f1a1406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.422+0000 7f1a2e371640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1a1406ca10 0x7f1a1406eed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009630 tx=0x7f1a1c009240 comp rx=0 tx=0).stop 2024-04-26T19:52:15.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.422+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a280b3740 msgr2=0x7f1a28156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.422+0000 7f1a2e371640 1 --2- 172.21.15.79:0/3584447358 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1a280b3740 0x7f1a28156920 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1a1802a1c0 tx=0x7f1a180046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.422+0000 7f1a2c90e640 1 -- 172.21.15.79:0/3584447358 reap_dead start 2024-04-26T19:52:15.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.422+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 shutdown_connections 2024-04-26T19:52:15.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.422+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 >> 172.21.15.79:0/3584447358 conn(0x7f1a28068390 msgr2=0x7f1a2815e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:15.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.422+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 shutdown_connections 2024-04-26T19:52:15.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.422+0000 7f1a2e371640 1 -- 172.21.15.79:0/3584447358 wait complete. 2024-04-26T19:52:15.433 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T19:52:15.530 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.530+0000 7f3e74789640 1 Processor -- start 2024-04-26T19:52:15.531 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.530+0000 7f3e74789640 1 -- start start 2024-04-26T19:52:15.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e74789640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 0x7f3e6c15d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e74789640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3e6c072a30 0x7f3e6c072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e74789640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f3e6c0b3de0 0x7f3e6c0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e74789640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f3e6c0734f0 con 0x7f3e6c072a30 2024-04-26T19:52:15.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e74789640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3e6c0737d0 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e74789640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6c0cde30 con 0x7f3e6c15d440 2024-04-26T19:52:15.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e71d24640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3e6c072a30 0x7f3e6c072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.532 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e71d24640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f3e6c072a30 0x7f3e6c072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44256/0 (socket says 172.21.15.79:44256) 2024-04-26T19:52:15.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e71d24640 1 -- 172.21.15.79:0/3748321368 learned_addr learned my addr 172.21.15.79:0/3748321368 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:15.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e72525640 1 --2- 172.21.15.79:0/3748321368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 0x7f3e6c15d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e72d26640 1 --2- 172.21.15.79:0/3748321368 >> v2:172.21.15.83:3300/0 conn(0x7f3e6c0b3de0 0x7f3e6c0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e71d24640 1 -- 172.21.15.79:0/3748321368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 msgr2=0x7f3e6c15d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:15.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e71d24640 1 --2- 172.21.15.79:0/3748321368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 0x7f3e6c15d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e71d24640 1 -- 172.21.15.79:0/3748321368 >> v2:172.21.15.83:3300/0 conn(0x7f3e6c0b3de0 msgr2=0x7f3e6c0b4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:15.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e71d24640 1 --2- 172.21.15.79:0/3748321368 >> v2:172.21.15.83:3300/0 conn(0x7f3e6c0b3de0 0x7f3e6c0b4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.533 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.531+0000 7f3e71d24640 1 -- 172.21.15.79:0/3748321368 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c0b3230 con 0x7f3e6c072a30 2024-04-26T19:52:15.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.532+0000 7f3e71d24640 1 --2- 172.21.15.79:0/3748321368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c072a30 0x7f3e6c072e90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0d1290 tx=0x7f3e5c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cfaf49bebd9d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.534 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.532+0000 7f3e71523640 1 -- 172.21.15.79:0/3748321368 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c037070 con 0x7f3e6c072a30 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.532+0000 7f3e71523640 1 -- 172.21.15.79:0/3748321368 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e5c032440 con 0x7f3e6c072a30 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.532+0000 7f3e71523640 1 -- 172.21.15.79:0/3748321368 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c032810 con 0x7f3e6c072a30 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.533+0000 7f3e74789640 1 -- 172.21.15.79:0/3748321368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c072a30 msgr2=0x7f3e6c072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.533+0000 7f3e74789640 1 --2- 172.21.15.79:0/3748321368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c072a30 0x7f3e6c072e90 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c0d1290 tx=0x7f3e5c02a1e0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.533+0000 7f3e74789640 1 -- 172.21.15.79:0/3748321368 shutdown_connections 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.533+0000 7f3e74789640 1 --2- 172.21.15.79:0/3748321368 >> v2:172.21.15.83:3300/0 conn(0x7f3e6c0b3de0 0x7f3e6c0b4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.533+0000 7f3e74789640 1 --2- 172.21.15.79:0/3748321368 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c072a30 0x7f3e6c072e90 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.533+0000 7f3e74789640 1 --2- 172.21.15.79:0/3748321368 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 0x7f3e6c15d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.533+0000 7f3e74789640 1 -- 172.21.15.79:0/3748321368 >> 172.21.15.79:0/3748321368 conn(0x7f3e6c0b39d0 msgr2=0x7f3e6c06feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.533+0000 7f3e74789640 1 -- 172.21.15.79:0/3748321368 shutdown_connections 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.533+0000 7f3e74789640 1 -- 172.21.15.79:0/3748321368 wait complete. 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.534+0000 7f3e74789640 1 Processor -- start 2024-04-26T19:52:15.535 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.534+0000 7f3e74789640 1 -- start start 2024-04-26T19:52:15.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e74789640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c072a30 0x7f3e6c160470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.537 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e74789640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c0b3de0 0x7f3e6c158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e74789640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 0x7f3e6c1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e74789640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c0ce630 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e74789640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6c15dd60 con 0x7f3e6c072a30 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e74789640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6c0ce7b0 con 0x7f3e6c15d440 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e71d24640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c0b3de0 0x7f3e6c158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e71d24640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c0b3de0 0x7f3e6c158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44272/0 (socket says 172.21.15.79:44272) 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e71d24640 1 -- 172.21.15.79:0/1436517216 learned_addr learned my addr 172.21.15.79:0/1436517216 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e72d26640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 0x7f3e6c1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e72525640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c072a30 0x7f3e6c160470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e71d24640 1 -- 172.21.15.79:0/1436517216 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 msgr2=0x7f3e6c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e71d24640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 0x7f3e6c1586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e71d24640 1 -- 172.21.15.79:0/1436517216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c072a30 msgr2=0x7f3e6c160470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.538 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e71d24640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c072a30 0x7f3e6c160470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e71d24640 1 -- 172.21.15.79:0/1436517216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6c1609b0 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e71d24640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c0b3de0 0x7f3e6c158160 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c02a220 tx=0x7f3e5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e62ffd640 1 -- 172.21.15.79:0/1436517216 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c041020 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e62ffd640 1 -- 172.21.15.79:0/1436517216 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e5c0306c0 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e6c156fa0 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.535+0000 7f3e62ffd640 1 -- 172.21.15.79:0/1436517216 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c037040 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.536+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e6c157560 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.536+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e6c0506c0 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.539 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.537+0000 7f3e62ffd640 1 -- 172.21.15.79:0/1436517216 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f3e5c044020 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.537+0000 7f3e62ffd640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3e5006ca10 0x7f3e5006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.538+0000 7f3e62ffd640 1 -- 172.21.15.79:0/1436517216 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f3e5c0ac600 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.538+0000 7f3e72525640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3e5006ca10 0x7f3e5006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.540 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.538+0000 7f3e72525640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3e5006ca10 0x7f3e5006eed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e640099c0 tx=0x7f3e64003990 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.542 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.542+0000 7f3e62ffd640 1 -- 172.21.15.79:0/1436517216 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f3e5c04b030 con 0x7f3e6c0b3de0 2024-04-26T19:52:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.716+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f3e6c0cf710 con 0x7f3e5006ca10 2024-04-26T19:52:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.717+0000 7f3e62ffd640 1 -- 172.21.15.79:0/1436517216 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f3e6c0cf710 con 0x7f3e5006ca10 2024-04-26T19:52:15.717 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T19:52:15.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.718+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3e5006ca10 msgr2=0x7f3e5006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.719 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.718+0000 7f3e74789640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3e5006ca10 0x7f3e5006eed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e640099c0 tx=0x7f3e64003990 comp rx=0 tx=0).stop 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c0b3de0 msgr2=0x7f3e6c158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c0b3de0 0x7f3e6c158160 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c02a220 tx=0x7f3e5c0046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 shutdown_connections 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f3e5006ca10 0x7f3e5006eed0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f3e6c15d440 0x7f3e6c1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f3e6c0b3de0 0x7f3e6c158160 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 --2- 172.21.15.79:0/1436517216 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f3e6c072a30 0x7f3e6c160470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 >> 172.21.15.79:0/1436517216 conn(0x7f3e6c0b39d0 msgr2=0x7f3e6c0d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 shutdown_connections 2024-04-26T19:52:15.720 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.719+0000 7f3e74789640 1 -- 172.21.15.79:0/1436517216 wait complete. 2024-04-26T19:52:15.730 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T19:52:15.730 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T19:52:15.829 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.829+0000 7f380fe99640 1 Processor -- start 2024-04-26T19:52:15.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.829+0000 7f380fe99640 1 -- start start 2024-04-26T19:52:15.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380fe99640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f380806cef0 0x7f380806f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380fe99640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380815fc60 con 0x7f380806caa0 2024-04-26T19:52:15.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380fe99640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380815ff40 con 0x7f3808148f40 2024-04-26T19:52:15.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380fe99640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f3808160220 con 0x7f380806cef0 2024-04-26T19:52:15.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380d434640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f380806caa0 0x7f380806f1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50930/0 (socket says 172.21.15.79:50930) 2024-04-26T19:52:15.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380d434640 1 -- 172.21.15.79:0/1445616898 learned_addr learned my addr 172.21.15.79:0/1445616898 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:15.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1440660846 0 0) 0x7f380815fc60 con 0x7f380806caa0 2024-04-26T19:52:15.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37f4003520 con 0x7f380806caa0 2024-04-26T19:52:15.833 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380e436640 1 --2- 172.21.15.79:0/1445616898 >> v2:172.21.15.83:3300/0 conn(0x7f380806cef0 0x7f380806f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1572443405 0 0) 0x7f37f4003520 con 0x7f380806caa0 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 >> v2:172.21.15.83:3300/0 conn(0x7f380806cef0 msgr2=0x7f380806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 --2- 172.21.15.79:0/1445616898 >> v2:172.21.15.83:3300/0 conn(0x7f380806cef0 0x7f380806f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 >> v1:172.21.15.83:6789/0 conn(0x7f3808148f40 legacy=0x7f38080d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3808160500 con 0x7f380806caa0 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f37f8001670 con 0x7f380806caa0 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 >> v1:172.21.15.79:6789/0 conn(0x7f380806caa0 legacy=0x7f380806f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.830+0000 7f380cc33640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37f40039b0 0x7f37f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380cc33640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4004800 0x7f37f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380cc33640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37f40051c0 0x7f37f400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f3808160220 con 0x7f37f40051c0 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f37f400c060 con 0x7f37f4004800 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f37f400c260 con 0x7f37f40039b0 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380e436640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37f40051c0 0x7f37f400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380dc35640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37f40039b0 0x7f37f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380d434640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4004800 0x7f37f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380e436640 1 -- 172.21.15.79:0/1445616898 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37f40039b0 msgr2=0x7f37f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380e436640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37f40039b0 0x7f37f4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380e436640 1 -- 172.21.15.79:0/1445616898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4004800 msgr2=0x7f37f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380e436640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f37f4004800 0x7f37f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.835 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380e436640 1 -- 172.21.15.79:0/1445616898 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f4003520 con 0x7f37f40051c0 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.831+0000 7f380e436640 1 -- 172.21.15.79:0/1445616898 reap_dead start 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380e436640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37f40051c0 0x7f37f400b9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3800009fc0 tx=0x7f37f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4836792bbfc199be server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800035020 con 0x7f37f40051c0 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3800003bd0 con 0x7f37f40051c0 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380cc33640 1 -- 172.21.15.79:0/1445616898 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800009c10 con 0x7f37f40051c0 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380fe99640 1 -- 172.21.15.79:0/1445616898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37f40051c0 msgr2=0x7f37f400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380fe99640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37f40051c0 0x7f37f400b9c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f3800009fc0 tx=0x7f37f4005830 comp rx=0 tx=0).stop 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380fe99640 1 -- 172.21.15.79:0/1445616898 shutdown_connections 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380fe99640 1 --2- 172.21.15.79:0/1445616898 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f37f40051c0 0x7f37f400b9c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380fe99640 1 -- 172.21.15.79:0/1445616898 >> 172.21.15.79:0/1445616898 conn(0x7f38080687e0 msgr2=0x7f38080d2340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380fe99640 1 -- 172.21.15.79:0/1445616898 shutdown_connections 2024-04-26T19:52:15.836 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.832+0000 7f380fe99640 1 -- 172.21.15.79:0/1445616898 wait complete. 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.833+0000 7f380fe99640 1 Processor -- start 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380fe99640 1 -- start start 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380fe99640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37f40051c0 0x7f38081599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380fe99640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38081eb080 0x7f38081eb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380fe99640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38081e9110 0x7f38081e95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380fe99640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f38080d1970 con 0x7f38081e9110 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380fe99640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f380806d340 con 0x7f38081eb080 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380fe99640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f3808149320 con 0x7f37f40051c0 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380e436640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38081e9110 0x7f38081e95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380e436640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38081e9110 0x7f38081e95c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44294/0 (socket says 172.21.15.79:44294) 2024-04-26T19:52:15.840 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380e436640 1 -- 172.21.15.79:0/3114372172 learned_addr learned my addr 172.21.15.79:0/3114372172 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380dc35640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37f40051c0 0x7f38081599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.834+0000 7f380d434640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38081eb080 0x7f38081eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f380e436640 1 -- 172.21.15.79:0/3114372172 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37f40051c0 msgr2=0x7f38081599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f380e436640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f37f40051c0 0x7f38081599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f380e436640 1 -- 172.21.15.79:0/3114372172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38081eb080 msgr2=0x7f38081eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f380e436640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f38081eb080 0x7f38081eb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f380e436640 1 -- 172.21.15.79:0/3114372172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3808159f00 con 0x7f38081e9110 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f380e436640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38081e9110 0x7f38081e95c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f380002af10 tx=0x7f3800009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f37fe7fc640 1 -- 172.21.15.79:0/3114372172 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800040070 con 0x7f38081e9110 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f37fe7fc640 1 -- 172.21.15.79:0/3114372172 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3800032e80 con 0x7f38081e9110 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f37fe7fc640 1 -- 172.21.15.79:0/3114372172 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800035030 con 0x7f38081e9110 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38081e9e80 con 0x7f38081e9110 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.835+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38081ea440 con 0x7f38081e9110 2024-04-26T19:52:15.841 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.836+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f38080506c0 con 0x7f38081e9110 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.836+0000 7f37fe7fc640 1 -- 172.21.15.79:0/3114372172 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f3800043070 con 0x7f38081e9110 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.837+0000 7f37fe7fc640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f37ec06ca40 0x7f37ec06ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.837+0000 7f37fe7fc640 1 -- 172.21.15.79:0/3114372172 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f38000ace60 con 0x7f38081e9110 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.837+0000 7f380dc35640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f37ec06ca40 0x7f37ec06ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.837+0000 7f37fe7fc640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7f37ec073430 0x7f37ec075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.837+0000 7f37fe7fc640 1 -- 172.21.15.79:0/3114372172 --> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f380003ce50 con 0x7f37ec073430 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.837+0000 7f37fe7fc640 1 -- 172.21.15.79:0/3114372172 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f38000af560 con 0x7f38081e9110 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.837+0000 7f380d434640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7f37ec073430 0x7f37ec075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.837+0000 7f380dc35640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f37ec06ca40 0x7f37ec06ef00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3804000c00 tx=0x7f3804009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.837+0000 7f380d434640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7f37ec073430 0x7f37ec075870 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.842 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.838+0000 7f37fe7fc640 1 -- 172.21.15.79:0/3114372172 <== osd.1 v2:172.21.15.79:6824/2915211239 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f380003ce50 con 0x7f37ec073430 2024-04-26T19:52:15.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.851+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 --> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f38080b0b70 con 0x7f37ec073430 2024-04-26T19:52:15.852 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.851+0000 7f37fe7fc640 1 -- 172.21.15.79:0/3114372172 <== osd.1 v2:172.21.15.79:6824/2915211239 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f38080b0b70 con 0x7f37ec073430 2024-04-26T19:52:15.852 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:52:15.852 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T19:52:15.852 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:52:15.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.851+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7f37ec073430 msgr2=0x7f37ec075870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.851+0000 7f380fe99640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6824/2915211239,v1:172.21.15.79:6825/2915211239] conn(0x7f37ec073430 0x7f37ec075870 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.852+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f37ec06ca40 msgr2=0x7f37ec06ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.852+0000 7f380fe99640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f37ec06ca40 0x7f37ec06ef00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3804000c00 tx=0x7f3804009040 comp rx=0 tx=0).stop 2024-04-26T19:52:15.853 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.852+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38081e9110 msgr2=0x7f38081e95c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.852+0000 7f380fe99640 1 --2- 172.21.15.79:0/3114372172 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f38081e9110 0x7f38081e95c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f380002af10 tx=0x7f3800009750 comp rx=0 tx=0).stop 2024-04-26T19:52:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.852+0000 7f380e436640 1 -- 172.21.15.79:0/3114372172 reap_dead start 2024-04-26T19:52:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.852+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 shutdown_connections 2024-04-26T19:52:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.852+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 >> 172.21.15.79:0/3114372172 conn(0x7f38080687e0 msgr2=0x7f3808073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.852+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 shutdown_connections 2024-04-26T19:52:15.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.852+0000 7f380fe99640 1 -- 172.21.15.79:0/3114372172 wait complete. 2024-04-26T19:52:15.862 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T19:52:15.960 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.960+0000 7f60345e7640 1 Processor -- start 2024-04-26T19:52:15.961 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.960+0000 7f60345e7640 1 -- start start 2024-04-26T19:52:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f60345e7640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 0x7f602c163aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f60345e7640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f602c06d4c0 0x7f602c0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f60345e7640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f602c0b4fc0 con 0x7f602c06d0e0 2024-04-26T19:52:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f60345e7640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f602c0b4d60 con 0x7f602c06d4c0 2024-04-26T19:52:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f60345e7640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f602c0c9130 con 0x7f602c148ed0 2024-04-26T19:52:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f6032b84640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f602c06d4c0 0x7f602c0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.962 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f6032b84640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f602c06d4c0 0x7f602c0b4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44310/0 (socket says 172.21.15.79:44310) 2024-04-26T19:52:15.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f6032b84640 1 -- 172.21.15.79:0/1444179933 learned_addr learned my addr 172.21.15.79:0/1444179933 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:15.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f6031b82640 1 --2- 172.21.15.79:0/1444179933 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 0x7f602c163aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f6032b84640 1 -- 172.21.15.79:0/1444179933 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 msgr2=0x7f602c163aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:15.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f6032b84640 1 --2- 172.21.15.79:0/1444179933 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 0x7f602c163aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f6032b84640 1 -- 172.21.15.79:0/1444179933 >> v1:172.21.15.83:6789/0 conn(0x7f602c06d0e0 legacy=0x7f602c1487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f6032b84640 1 -- 172.21.15.79:0/1444179933 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602c0c9410 con 0x7f602c06d4c0 2024-04-26T19:52:15.963 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.961+0000 7f6031b82640 1 --2- 172.21.15.79:0/1444179933 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 0x7f602c163aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:15.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.962+0000 7f6032b84640 1 --2- 172.21.15.79:0/1444179933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 0x7f602c0b4820 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6024009e50 tx=0x7f602402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a0bd83eb7305ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.964 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.962+0000 7f6031381640 1 -- 172.21.15.79:0/1444179933 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024036070 con 0x7f602c06d4c0 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.962+0000 7f6031381640 1 -- 172.21.15.79:0/1444179933 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6024031600 con 0x7f602c06d4c0 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.962+0000 7f6031381640 1 -- 172.21.15.79:0/1444179933 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60240319d0 con 0x7f602c06d4c0 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.963+0000 7f60345e7640 1 -- 172.21.15.79:0/1444179933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 msgr2=0x7f602c0b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.963+0000 7f60345e7640 1 --2- 172.21.15.79:0/1444179933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 0x7f602c0b4820 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6024009e50 tx=0x7f602402a680 comp rx=0 tx=0).stop 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.963+0000 7f60345e7640 1 -- 172.21.15.79:0/1444179933 shutdown_connections 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.963+0000 7f60345e7640 1 --2- 172.21.15.79:0/1444179933 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 0x7f602c0b4820 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.963+0000 7f60345e7640 1 --2- 172.21.15.79:0/1444179933 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 0x7f602c163aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.963+0000 7f60345e7640 1 -- 172.21.15.79:0/1444179933 >> 172.21.15.79:0/1444179933 conn(0x7f602c06c350 msgr2=0x7f602c06c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.963+0000 7f60345e7640 1 -- 172.21.15.79:0/1444179933 shutdown_connections 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.963+0000 7f60345e7640 1 -- 172.21.15.79:0/1444179933 wait complete. 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.964+0000 7f60345e7640 1 Processor -- start 2024-04-26T19:52:15.965 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.964+0000 7f60345e7640 1 -- start start 2024-04-26T19:52:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f60345e7640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c06d0e0 0x7f602c1f4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f60345e7640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 0x7f602c1f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f60345e7640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 0x7f602c0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f60345e7640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f602c0cc510 con 0x7f602c06d4c0 2024-04-26T19:52:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f60345e7640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f602c0c99d0 con 0x7f602c06d0e0 2024-04-26T19:52:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f60345e7640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f602c0cc690 con 0x7f602c148ed0 2024-04-26T19:52:15.968 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6031b82640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 0x7f602c1f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6031b82640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 0x7f602c1f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44320/0 (socket says 172.21.15.79:44320) 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6031b82640 1 -- 172.21.15.79:0/55863097 learned_addr learned my addr 172.21.15.79:0/55863097 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6032383640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c06d0e0 0x7f602c1f4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6032b84640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 0x7f602c0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6031b82640 1 -- 172.21.15.79:0/55863097 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 msgr2=0x7f602c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6031b82640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 0x7f602c0ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6031b82640 1 -- 172.21.15.79:0/55863097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c06d0e0 msgr2=0x7f602c1f4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6031b82640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c06d0e0 0x7f602c1f4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6031b82640 1 -- 172.21.15.79:0/55863097 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f602c1f9390 con 0x7f602c06d4c0 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6032383640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c06d0e0 0x7f602c1f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6031b82640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 0x7f602c1f8e50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f601c00d830 tx=0x7f601c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6022ffd640 1 -- 172.21.15.79:0/55863097 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c018020 con 0x7f602c06d4c0 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6022ffd640 1 -- 172.21.15.79:0/55863097 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f601c004030 con 0x7f602c06d4c0 2024-04-26T19:52:15.969 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f602c1f5460 con 0x7f602c06d4c0 2024-04-26T19:52:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f6022ffd640 1 -- 172.21.15.79:0/55863097 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f601c004420 con 0x7f602c06d4c0 2024-04-26T19:52:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.965+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f602c1f5a20 con 0x7f602c06d4c0 2024-04-26T19:52:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.966+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f602c0506c0 con 0x7f602c06d4c0 2024-04-26T19:52:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.967+0000 7f6022ffd640 1 -- 172.21.15.79:0/55863097 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f601c005ba0 con 0x7f602c06d4c0 2024-04-26T19:52:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.967+0000 7f6022ffd640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f601006ca10 0x7f601006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.967+0000 7f6022ffd640 1 -- 172.21.15.79:0/55863097 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f601c08c030 con 0x7f602c06d4c0 2024-04-26T19:52:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.967+0000 7f6032383640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f601006ca10 0x7f601006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:15.970 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.967+0000 7f6032383640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f601006ca10 0x7f601006eed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6028001700 tx=0x7f6028009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:15.972 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:15.971+0000 7f6022ffd640 1 -- 172.21.15.79:0/55863097 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f601c022020 con 0x7f602c06d4c0 2024-04-26T19:52:16.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.146+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f602c162d20 con 0x7f601006ca10 2024-04-26T19:52:16.147 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.146+0000 7f6022ffd640 1 -- 172.21.15.79:0/55863097 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f602c162d20 con 0x7f601006ca10 2024-04-26T19:52:16.147 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f601006ca10 msgr2=0x7f601006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f601006ca10 0x7f601006eed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6028001700 tx=0x7f6028009040 comp rx=0 tx=0).stop 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 msgr2=0x7f602c1f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 0x7f602c1f8e50 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f601c00d830 tx=0x7f601c00dcf0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 shutdown_connections 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f601006ca10 0x7f601006eed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f602c148ed0 0x7f602c0ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f602c06d4c0 0x7f602c1f8e50 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 --2- 172.21.15.79:0/55863097 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f602c06d0e0 0x7f602c1f4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 >> 172.21.15.79:0/55863097 conn(0x7f602c06c350 msgr2=0x7f602c06c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 shutdown_connections 2024-04-26T19:52:16.149 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.148+0000 7f60345e7640 1 -- 172.21.15.79:0/55863097 wait complete. 2024-04-26T19:52:16.159 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T19:52:16.160 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T19:52:16.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.262+0000 7fbc21589640 1 Processor -- start 2024-04-26T19:52:16.263 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.262+0000 7fbc21589640 1 -- start start 2024-04-26T19:52:16.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.262+0000 7fbc21589640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbc1c159b30 0x7fbc1c06f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc21589640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fbc1c06df80 0x7fbc1c160560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc21589640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc1c160aa0 con 0x7fbc1c144ca0 2024-04-26T19:52:16.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc21589640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1c145bb0 con 0x7fbc1c159b30 2024-04-26T19:52:16.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc21589640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fbc1c06f090 con 0x7fbc1c06df80 2024-04-26T19:52:16.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc1affd640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbc1c159b30 0x7fbc1c06f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc1b7fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fbc1c144ca0 0x7fbc1c1450e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50944/0 (socket says 172.21.15.79:50944) 2024-04-26T19:52:16.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc1b7fe640 1 -- 172.21.15.79:0/4289748070 learned_addr learned my addr 172.21.15.79:0/4289748070 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc1a7fc640 1 --2- 172.21.15.79:0/4289748070 >> v2:172.21.15.83:3300/0 conn(0x7fbc1c06df80 0x7fbc1c160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc1affd640 1 -- 172.21.15.79:0/4289748070 >> v2:172.21.15.83:3300/0 conn(0x7fbc1c06df80 msgr2=0x7fbc1c160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc19ffb640 1 -- 172.21.15.79:0/4289748070 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3620051355 0 0) 0x7fbc14001490 con 0x7fbc1c144ca0 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc1affd640 1 --2- 172.21.15.79:0/4289748070 >> v2:172.21.15.83:3300/0 conn(0x7fbc1c06df80 0x7fbc1c160560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc1affd640 1 -- 172.21.15.79:0/4289748070 >> v1:172.21.15.79:6789/0 conn(0x7fbc1c144ca0 legacy=0x7fbc1c1450e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc1affd640 1 -- 172.21.15.79:0/4289748070 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c06e480 con 0x7fbc1c159b30 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.263+0000 7fbc1a7fc640 1 --2- 172.21.15.79:0/4289748070 >> v2:172.21.15.83:3300/0 conn(0x7fbc1c06df80 0x7fbc1c160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc1affd640 1 --2- 172.21.15.79:0/4289748070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c159b30 0x7fbc1c06f520 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c06f210 tx=0x7fbc0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c489b2c2464a4c2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc19ffb640 1 -- 172.21.15.79:0/4289748070 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c037070 con 0x7fbc1c159b30 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc19ffb640 1 -- 172.21.15.79:0/4289748070 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0c032440 con 0x7fbc1c159b30 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc19ffb640 1 -- 172.21.15.79:0/4289748070 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c032810 con 0x7fbc1c159b30 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc21589640 1 -- 172.21.15.79:0/4289748070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c159b30 msgr2=0x7fbc1c06f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.266 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc21589640 1 --2- 172.21.15.79:0/4289748070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c159b30 0x7fbc1c06f520 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c06f210 tx=0x7fbc0c02a1a0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc21589640 1 -- 172.21.15.79:0/4289748070 shutdown_connections 2024-04-26T19:52:16.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc21589640 1 --2- 172.21.15.79:0/4289748070 >> v2:172.21.15.83:3300/0 conn(0x7fbc1c06df80 0x7fbc1c160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc21589640 1 --2- 172.21.15.79:0/4289748070 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c159b30 0x7fbc1c06f520 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc21589640 1 -- 172.21.15.79:0/4289748070 >> 172.21.15.79:0/4289748070 conn(0x7fbc1c0685d0 msgr2=0x7fbc1c0d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:16.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc21589640 1 -- 172.21.15.79:0/4289748070 shutdown_connections 2024-04-26T19:52:16.267 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.264+0000 7fbc21589640 1 -- 172.21.15.79:0/4289748070 wait complete. 2024-04-26T19:52:16.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.266+0000 7fbc21589640 1 Processor -- start 2024-04-26T19:52:16.270 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.266+0000 7fbc21589640 1 -- start start 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.266+0000 7fbc21589640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc1c06df80 0x7fbc1c14aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.266+0000 7fbc21589640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c144ca0 0x7fbc1c140140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.266+0000 7fbc21589640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc1c159b30 0x7fbc1c1f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.266+0000 7fbc21589640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c06ed20 con 0x7fbc1c144ca0 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.266+0000 7fbc21589640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbc1c06ea40 con 0x7fbc1c06df80 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.266+0000 7fbc21589640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbc1c159f10 con 0x7fbc1c159b30 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1a7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c144ca0 0x7fbc1c140140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1a7fc640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c144ca0 0x7fbc1c140140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44338/0 (socket says 172.21.15.79:44338) 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1a7fc640 1 -- 172.21.15.79:0/2745847104 learned_addr learned my addr 172.21.15.79:0/2745847104 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1b7fe640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc1c159b30 0x7fbc1c1f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1affd640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc1c06df80 0x7fbc1c14aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1a7fc640 1 -- 172.21.15.79:0/2745847104 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc1c159b30 msgr2=0x7fbc1c1f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1a7fc640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbc1c159b30 0x7fbc1c1f8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.271 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1a7fc640 1 -- 172.21.15.79:0/2745847104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc1c06df80 msgr2=0x7fbc1c14aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1a7fc640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc1c06df80 0x7fbc1c14aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1a7fc640 1 -- 172.21.15.79:0/2745847104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc1c140680 con 0x7fbc1c144ca0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1affd640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbc1c06df80 0x7fbc1c14aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc1a7fc640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c144ca0 0x7fbc1c140140 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c7c0 tx=0x7fbc1000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc037fe640 1 -- 172.21.15.79:0/2745847104 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10015070 con 0x7fbc1c144ca0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc037fe640 1 -- 172.21.15.79:0/2745847104 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc10007820 con 0x7fbc1c144ca0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc1c0affb0 con 0x7fbc1c144ca0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc037fe640 1 -- 172.21.15.79:0/2745847104 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10007bf0 con 0x7fbc1c144ca0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.267+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc1c1f8fa0 con 0x7fbc1c144ca0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.268+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbc1c0506c0 con 0x7fbc1c144ca0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.269+0000 7fbc037fe640 1 -- 172.21.15.79:0/2745847104 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7fbc10018070 con 0x7fbc1c144ca0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.269+0000 7fbc037fe640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbc0806ca40 0x7fbc0806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.269+0000 7fbc037fe640 1 -- 172.21.15.79:0/2745847104 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fbc1008bca0 con 0x7fbc1c144ca0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.269+0000 7fbc1affd640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbc0806ca40 0x7fbc0806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.272 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.269+0000 7fbc037fe640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fbc08073430 0x7fbc08075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.269+0000 7fbc037fe640 1 -- 172.21.15.79:0/2745847104 --> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc08075f40 con 0x7fbc08073430 2024-04-26T19:52:16.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.269+0000 7fbc037fe640 1 -- 172.21.15.79:0/2745847104 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbc1008e3a0 con 0x7fbc1c144ca0 2024-04-26T19:52:16.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.269+0000 7fbc1b7fe640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fbc08073430 0x7fbc08075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.269+0000 7fbc1affd640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbc0806ca40 0x7fbc0806ef00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c034000 tx=0x7fbc0c02e000 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.270+0000 7fbc1b7fe640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fbc08073430 0x7fbc08075870 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.273 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.270+0000 7fbc037fe640 1 -- 172.21.15.79:0/2745847104 <== osd.2 v2:172.21.15.79:6808/569648363 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbc08075f40 con 0x7fbc08073430 2024-04-26T19:52:16.283 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.283+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 --> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbc1c1422f0 con 0x7fbc08073430 2024-04-26T19:52:16.284 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.283+0000 7fbc037fe640 1 -- 172.21.15.79:0/2745847104 <== osd.2 v2:172.21.15.79:6808/569648363 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbc1c1422f0 con 0x7fbc08073430 2024-04-26T19:52:16.284 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:52:16.284 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T19:52:16.284 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:52:16.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fbc08073430 msgr2=0x7fbc08075870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6808/569648363,v1:172.21.15.79:6809/569648363] conn(0x7fbc08073430 0x7fbc08075870 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbc0806ca40 msgr2=0x7fbc0806ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbc0806ca40 0x7fbc0806ef00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c034000 tx=0x7fbc0c02e000 comp rx=0 tx=0).stop 2024-04-26T19:52:16.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c144ca0 msgr2=0x7fbc1c140140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.285 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 --2- 172.21.15.79:0/2745847104 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbc1c144ca0 0x7fbc1c140140 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbc1000c7c0 tx=0x7fbc1000cc80 comp rx=0 tx=0).stop 2024-04-26T19:52:16.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc1b7fe640 1 -- 172.21.15.79:0/2745847104 reap_dead start 2024-04-26T19:52:16.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 shutdown_connections 2024-04-26T19:52:16.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 >> 172.21.15.79:0/2745847104 conn(0x7fbc1c0685d0 msgr2=0x7fbc1c15f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:16.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 shutdown_connections 2024-04-26T19:52:16.286 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.284+0000 7fbc21589640 1 -- 172.21.15.79:0/2745847104 wait complete. 2024-04-26T19:52:16.296 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T19:52:16.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.393+0000 7fa0bf567640 1 Processor -- start 2024-04-26T19:52:16.394 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.394+0000 7fa0bf567640 1 -- start start 2024-04-26T19:52:16.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.394+0000 7fa0bf567640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b8159c70 0x7fa0b8163a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.394+0000 7fa0bf567640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b8160280 con 0x7fa0b80b3b40 2024-04-26T19:52:16.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.394+0000 7fa0bf567640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0b8160560 con 0x7fa0b8159890 2024-04-26T19:52:16.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.394+0000 7fa0bf567640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b8160840 con 0x7fa0b8159c70 2024-04-26T19:52:16.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0bdb04640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fa0b80b3b40 0x7fa0b8161340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50956/0 (socket says 172.21.15.79:50956) 2024-04-26T19:52:16.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0bdb04640 1 -- 172.21.15.79:0/3847883925 learned_addr learned my addr 172.21.15.79:0/3847883925 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:16.398 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1450975106 0 0) 0x7fa0b8160280 con 0x7fa0b80b3b40 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0a4003520 con 0x7fa0b80b3b40 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0bcb02640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b8159c70 0x7fa0b8163a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3307023411 0 0) 0x7fa0a4003520 con 0x7fa0b80b3b40 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b8159c70 msgr2=0x7fa0b8163a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b8159c70 0x7fa0b8163a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 >> v1:172.21.15.83:6789/0 conn(0x7fa0b8159890 legacy=0x7fa0b815fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b8160e20 con 0x7fa0b80b3b40 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fa0b00019f0 con 0x7fa0b80b3b40 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 >> v1:172.21.15.79:6789/0 conn(0x7fa0b80b3b40 legacy=0x7fa0b8161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a40039e0 0x7fa0a4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0a4004800 0x7fa0a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b8159c70 0x7fa0a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b8160840 con 0x7fa0a40039e0 2024-04-26T19:52:16.399 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a400ba70 con 0x7fa0a4004800 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa0a400bc70 con 0x7fa0b8159c70 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0bd303640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a40039e0 0x7fa0a4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0bdb04640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b8159c70 0x7fa0a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.395+0000 7fa0bcb02640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0a4004800 0x7fa0a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.396+0000 7fa0bd303640 1 -- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b8159c70 msgr2=0x7fa0a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.396+0000 7fa0bd303640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b8159c70 0x7fa0a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.396+0000 7fa0bd303640 1 -- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0a4004800 msgr2=0x7fa0a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.396+0000 7fa0bd303640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0a4004800 0x7fa0a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.396+0000 7fa0bd303640 1 -- 172.21.15.79:0/3847883925 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a4003520 con 0x7fa0a40039e0 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.396+0000 7fa0bd303640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a40039e0 0x7fa0a4003e00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8001720 tx=0x7fa0a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ff6ddf3b11357a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.396+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8001370 con 0x7fa0a40039e0 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.396+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0a80052d0 con 0x7fa0a40039e0 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.396+0000 7fa0affff640 1 -- 172.21.15.79:0/3847883925 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a802ab90 con 0x7fa0a40039e0 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.397+0000 7fa0bf567640 1 -- 172.21.15.79:0/3847883925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a40039e0 msgr2=0x7fa0a4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.400 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.397+0000 7fa0bf567640 1 --2- 172.21.15.79:0/3847883925 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0a40039e0 0x7fa0a4003e00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8001720 tx=0x7fa0a4005830 comp rx=0 tx=0).stop 2024-04-26T19:52:16.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.397+0000 7fa0bdb04640 1 -- 172.21.15.79:0/3847883925 reap_dead start 2024-04-26T19:52:16.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.397+0000 7fa0bf567640 1 -- 172.21.15.79:0/3847883925 shutdown_connections 2024-04-26T19:52:16.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.397+0000 7fa0bf567640 1 -- 172.21.15.79:0/3847883925 >> 172.21.15.79:0/3847883925 conn(0x7fa0b80b2720 msgr2=0x7fa0b8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:16.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.397+0000 7fa0bf567640 1 -- 172.21.15.79:0/3847883925 shutdown_connections 2024-04-26T19:52:16.401 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.397+0000 7fa0bf567640 1 -- 172.21.15.79:0/3847883925 wait complete. 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.398+0000 7fa0bf567640 1 Processor -- start 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.398+0000 7fa0bf567640 1 -- start start 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bf567640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0a40039e0 0x7fa0b81f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bf567640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0b80b01b0 0x7fa0b80b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bf567640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b815bf40 0x7fa0b815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bf567640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b8163bd0 con 0x7fa0b80b01b0 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bf567640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b8163a50 con 0x7fa0a40039e0 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bf567640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fa0b8163d50 con 0x7fa0b815bf40 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bcb02640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0b80b01b0 0x7fa0b80b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.404 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bcb02640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0b80b01b0 0x7fa0b80b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44358/0 (socket says 172.21.15.79:44358) 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bcb02640 1 -- 172.21.15.79:0/1880280603 learned_addr learned my addr 172.21.15.79:0/1880280603 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bdb04640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b815bf40 0x7fa0b815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bd303640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0a40039e0 0x7fa0b81f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bcb02640 1 -- 172.21.15.79:0/1880280603 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b815bf40 msgr2=0x7fa0b815c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bcb02640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b815bf40 0x7fa0b815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bcb02640 1 -- 172.21.15.79:0/1880280603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0a40039e0 msgr2=0x7fa0b81f4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bcb02640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0a40039e0 0x7fa0b81f4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bcb02640 1 -- 172.21.15.79:0/1880280603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b81f81b0 con 0x7fa0b80b01b0 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.399+0000 7fa0bcb02640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0b80b01b0 0x7fa0b80b0630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40119c0 tx=0x7fa0b4011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.400+0000 7fa0adffb640 1 -- 172.21.15.79:0/1880280603 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4019070 con 0x7fa0b80b01b0 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.400+0000 7fa0adffb640 1 -- 172.21.15.79:0/1880280603 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0b4004240 con 0x7fa0b80b01b0 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.400+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b81f8440 con 0x7fa0b80b01b0 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.400+0000 7fa0adffb640 1 -- 172.21.15.79:0/1880280603 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b400f7c0 con 0x7fa0b80b01b0 2024-04-26T19:52:16.405 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.400+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b81f8a00 con 0x7fa0b80b01b0 2024-04-26T19:52:16.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.401+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b80506c0 con 0x7fa0b80b01b0 2024-04-26T19:52:16.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.401+0000 7fa0adffb640 1 -- 172.21.15.79:0/1880280603 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7fa0b401c070 con 0x7fa0b80b01b0 2024-04-26T19:52:16.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.402+0000 7fa0adffb640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fa09c06ca10 0x7fa09c06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.402+0000 7fa0adffb640 1 -- 172.21.15.79:0/1880280603 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fa0b408c110 con 0x7fa0b80b01b0 2024-04-26T19:52:16.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.402+0000 7fa0bd303640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fa09c06ca10 0x7fa09c06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.402+0000 7fa0bd303640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fa09c06ca10 0x7fa09c06eed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8032000 tx=0x7fa0a802e000 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.406 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.406+0000 7fa0adffb640 1 -- 172.21.15.79:0/1880280603 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fa0b405a340 con 0x7fa0b80b01b0 2024-04-26T19:52:16.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.579+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fa0b8159140 con 0x7fa09c06ca10 2024-04-26T19:52:16.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.579+0000 7fa0adffb640 1 -- 172.21.15.79:0/1880280603 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa0b8159140 con 0x7fa09c06ca10 2024-04-26T19:52:16.580 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T19:52:16.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fa09c06ca10 msgr2=0x7fa09c06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fa09c06ca10 0x7fa09c06eed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa0a8032000 tx=0x7fa0a802e000 comp rx=0 tx=0).stop 2024-04-26T19:52:16.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0b80b01b0 msgr2=0x7fa0b80b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0b80b01b0 0x7fa0b80b0630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40119c0 tx=0x7fa0b4011e80 comp rx=0 tx=0).stop 2024-04-26T19:52:16.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 shutdown_connections 2024-04-26T19:52:16.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fa09c06ca10 0x7fa09c06eed0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.582 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fa0b815bf40 0x7fa0b815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fa0b80b01b0 0x7fa0b80b0630 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 --2- 172.21.15.79:0/1880280603 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fa0a40039e0 0x7fa0b81f4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 >> 172.21.15.79:0/1880280603 conn(0x7fa0b80b2720 msgr2=0x7fa0b815e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:16.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 shutdown_connections 2024-04-26T19:52:16.583 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.581+0000 7fa0bf567640 1 -- 172.21.15.79:0/1880280603 wait complete. 2024-04-26T19:52:16.592 INFO:tasks.ceph:Scrubbing osd.3 2024-04-26T19:52:16.593 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-26T19:52:16.693 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.691+0000 7fbe34132640 1 Processor -- start 2024-04-26T19:52:16.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.692+0000 7fbe34132640 1 -- start start 2024-04-26T19:52:16.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.692+0000 7fbe34132640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbe2c06dac0 0x7fbe2c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.692+0000 7fbe34132640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbe2c0b4b40 0x7fbe2c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.692+0000 7fbe34132640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe2c0d2b20 con 0x7fbe2c0b7f50 2024-04-26T19:52:16.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.692+0000 7fbe34132640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fbe2c0b8330 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.692+0000 7fbe34132640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2c0cfaf0 con 0x7fbe2c06dac0 2024-04-26T19:52:16.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.693+0000 7fbe326cf640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbe2c0b4b40 0x7fbe2c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.693+0000 7fbe326cf640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fbe2c0b4b40 0x7fbe2c0b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44374/0 (socket says 172.21.15.79:44374) 2024-04-26T19:52:16.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.693+0000 7fbe326cf640 1 -- 172.21.15.79:0/3686219340 learned_addr learned my addr 172.21.15.79:0/3686219340 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:16.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.693+0000 7fbe316cd640 1 --2- 172.21.15.79:0/3686219340 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbe2c06dac0 0x7fbe2c06df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.693+0000 7fbe326cf640 1 -- 172.21.15.79:0/3686219340 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbe2c06dac0 msgr2=0x7fbe2c06df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:16.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.693+0000 7fbe326cf640 1 --2- 172.21.15.79:0/3686219340 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbe2c06dac0 0x7fbe2c06df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.693+0000 7fbe326cf640 1 -- 172.21.15.79:0/3686219340 >> v1:172.21.15.83:6789/0 conn(0x7fbe2c0b7f50 legacy=0x7fbe2c0d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.693+0000 7fbe326cf640 1 -- 172.21.15.79:0/3686219340 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c0d2e00 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe326cf640 1 --2- 172.21.15.79:0/3686219340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 0x7fbe2c0b4fd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009e50 tx=0x7fbe2402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29303a6d96047a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe30ecc640 1 -- 172.21.15.79:0/3686219340 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe24036070 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe30ecc640 1 -- 172.21.15.79:0/3686219340 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe24031600 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe30ecc640 1 -- 172.21.15.79:0/3686219340 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe240319d0 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe34132640 1 -- 172.21.15.79:0/3686219340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 msgr2=0x7fbe2c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe34132640 1 --2- 172.21.15.79:0/3686219340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 0x7fbe2c0b4fd0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009e50 tx=0x7fbe2402a5f0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe34132640 1 -- 172.21.15.79:0/3686219340 shutdown_connections 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe34132640 1 --2- 172.21.15.79:0/3686219340 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 0x7fbe2c0b4fd0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe34132640 1 --2- 172.21.15.79:0/3686219340 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbe2c06dac0 0x7fbe2c06df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe34132640 1 -- 172.21.15.79:0/3686219340 >> 172.21.15.79:0/3686219340 conn(0x7fbe2c071a60 msgr2=0x7fbe2c071e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe34132640 1 -- 172.21.15.79:0/3686219340 shutdown_connections 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.694+0000 7fbe34132640 1 -- 172.21.15.79:0/3686219340 wait complete. 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.695+0000 7fbe34132640 1 Processor -- start 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe34132640 1 -- start start 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe34132640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe2c06dac0 0x7fbe2c13bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe34132640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 0x7fbe2c13eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe34132640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbe2c0b7f50 0x7fbe2c13f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe34132640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c0b3ea0 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe34132640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fbe2c0cfd40 con 0x7fbe2c06dac0 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe34132640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fbe2c0b4020 con 0x7fbe2c0b7f50 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe316cd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 0x7fbe2c13eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe316cd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 0x7fbe2c13eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44376/0 (socket says 172.21.15.79:44376) 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe316cd640 1 -- 172.21.15.79:0/3850492901 learned_addr learned my addr 172.21.15.79:0/3850492901 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:16.699 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe326cf640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbe2c0b7f50 0x7fbe2c13f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.696+0000 7fbe31ece640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe2c06dac0 0x7fbe2c13bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe316cd640 1 -- 172.21.15.79:0/3850492901 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbe2c0b7f50 msgr2=0x7fbe2c13f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe316cd640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fbe2c0b7f50 0x7fbe2c13f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe316cd640 1 -- 172.21.15.79:0/3850492901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe2c06dac0 msgr2=0x7fbe2c13bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe316cd640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fbe2c06dac0 0x7fbe2c13bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe316cd640 1 -- 172.21.15.79:0/3850492901 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2c13f920 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe316cd640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 0x7fbe2c13eb60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e9b0 tx=0x7fbe2800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe227fc640 1 -- 172.21.15.79:0/3850492901 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe28017070 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe227fc640 1 -- 172.21.15.79:0/3850492901 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe2800c9a0 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2c13fc00 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe227fc640 1 -- 172.21.15.79:0/3850492901 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2800cd90 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.700 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.697+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe2c140510 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.698+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe2c0b72d0 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.699+0000 7fbe227fc640 1 -- 172.21.15.79:0/3850492901 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7fbe2801a070 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.699+0000 7fbe227fc640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbe1006ca10 0x7fbe1006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.699+0000 7fbe227fc640 1 -- 172.21.15.79:0/3850492901 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fbe2808c0c0 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.699+0000 7fbe31ece640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbe1006ca10 0x7fbe1006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.699+0000 7fbe227fc640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbe10073400 0x7fbe10075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.699+0000 7fbe227fc640 1 -- 172.21.15.79:0/3850492901 --> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe2801fe50 con 0x7fbe10073400 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.699+0000 7fbe227fc640 1 -- 172.21.15.79:0/3850492901 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbe28093050 con 0x7fbe2c0b4b40 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.699+0000 7fbe326cf640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbe10073400 0x7fbe10075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.702 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.699+0000 7fbe31ece640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbe1006ca10 0x7fbe1006eed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c148460 tx=0x7fbe1c009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.700+0000 7fbe326cf640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbe10073400 0x7fbe10075840 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.703 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.700+0000 7fbe227fc640 1 -- 172.21.15.79:0/3850492901 <== osd.3 v2:172.21.15.79:6816/1557374359 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbe2801fe50 con 0x7fbe10073400 2024-04-26T19:52:16.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.713+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 --> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbe2c0506c0 con 0x7fbe10073400 2024-04-26T19:52:16.714 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.713+0000 7fbe227fc640 1 -- 172.21.15.79:0/3850492901 <== osd.3 v2:172.21.15.79:6816/1557374359 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbe2c0506c0 con 0x7fbe10073400 2024-04-26T19:52:16.714 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:52:16.714 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T19:52:16.714 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:52:16.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbe10073400 msgr2=0x7fbe10075840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6816/1557374359,v1:172.21.15.79:6817/1557374359] conn(0x7fbe10073400 0x7fbe10075840 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.715 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbe1006ca10 msgr2=0x7fbe1006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fbe1006ca10 0x7fbe1006eed0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c148460 tx=0x7fbe1c009040 comp rx=0 tx=0).stop 2024-04-26T19:52:16.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 msgr2=0x7fbe2c13eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 --2- 172.21.15.79:0/3850492901 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fbe2c0b4b40 0x7fbe2c13eb60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800e9b0 tx=0x7fbe2800ee70 comp rx=0 tx=0).stop 2024-04-26T19:52:16.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe326cf640 1 -- 172.21.15.79:0/3850492901 reap_dead start 2024-04-26T19:52:16.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 shutdown_connections 2024-04-26T19:52:16.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 >> 172.21.15.79:0/3850492901 conn(0x7fbe2c071a60 msgr2=0x7fbe2c06e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:16.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 shutdown_connections 2024-04-26T19:52:16.716 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.714+0000 7fbe34132640 1 -- 172.21.15.79:0/3850492901 wait complete. 2024-04-26T19:52:16.725 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-26T19:52:16.823 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.822+0000 7fba7bc83640 1 Processor -- start 2024-04-26T19:52:16.824 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.822+0000 7fba7bc83640 1 -- start start 2024-04-26T19:52:16.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7bc83640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fba74160390 0x7fba740b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7bc83640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fba741637e0 0x7fba740b4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7bc83640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba74144ca0 con 0x7fba74160770 2024-04-26T19:52:16.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7bc83640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fba74163e00 con 0x7fba741637e0 2024-04-26T19:52:16.826 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7bc83640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fba74144f30 con 0x7fba74160390 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7a220640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fba741637e0 0x7fba740b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7a220640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fba741637e0 0x7fba740b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44386/0 (socket says 172.21.15.79:44386) 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7a220640 1 -- 172.21.15.79:0/2005668371 learned_addr learned my addr 172.21.15.79:0/2005668371 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba79a1f640 1 --2- 172.21.15.79:0/2005668371 >> v2:172.21.15.83:3300/0 conn(0x7fba74160390 0x7fba740b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7a220640 1 -- 172.21.15.79:0/2005668371 >> v2:172.21.15.83:3300/0 conn(0x7fba74160390 msgr2=0x7fba740b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7a220640 1 --2- 172.21.15.79:0/2005668371 >> v2:172.21.15.83:3300/0 conn(0x7fba74160390 0x7fba740b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7a220640 1 -- 172.21.15.79:0/2005668371 >> v1:172.21.15.83:6789/0 conn(0x7fba74160770 legacy=0x7fba741630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.823+0000 7fba7a220640 1 -- 172.21.15.79:0/2005668371 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba74145210 con 0x7fba741637e0 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba7a220640 1 --2- 172.21.15.79:0/2005668371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 0x7fba740b4050 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fba740b5690 tx=0x7fba6c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fd4c2f8135a6b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba78a1d640 1 -- 172.21.15.79:0/2005668371 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c036070 con 0x7fba741637e0 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba78a1d640 1 -- 172.21.15.79:0/2005668371 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c031600 con 0x7fba741637e0 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba78a1d640 1 -- 172.21.15.79:0/2005668371 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0319d0 con 0x7fba741637e0 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba7bc83640 1 -- 172.21.15.79:0/2005668371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 msgr2=0x7fba740b4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.827 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba7bc83640 1 --2- 172.21.15.79:0/2005668371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 0x7fba740b4050 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fba740b5690 tx=0x7fba6c02a6a0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba7bc83640 1 -- 172.21.15.79:0/2005668371 shutdown_connections 2024-04-26T19:52:16.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba7bc83640 1 --2- 172.21.15.79:0/2005668371 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 0x7fba740b4050 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba7bc83640 1 --2- 172.21.15.79:0/2005668371 >> v2:172.21.15.83:3300/0 conn(0x7fba74160390 0x7fba740b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.824+0000 7fba7bc83640 1 -- 172.21.15.79:0/2005668371 >> 172.21.15.79:0/2005668371 conn(0x7fba740b2720 msgr2=0x7fba74070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:16.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.825+0000 7fba7bc83640 1 -- 172.21.15.79:0/2005668371 shutdown_connections 2024-04-26T19:52:16.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.825+0000 7fba7bc83640 1 -- 172.21.15.79:0/2005668371 wait complete. 2024-04-26T19:52:16.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.826+0000 7fba7bc83640 1 Processor -- start 2024-04-26T19:52:16.828 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.826+0000 7fba7bc83640 1 -- start start 2024-04-26T19:52:16.830 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.826+0000 7fba7bc83640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fba74160390 0x7fba74149490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.826+0000 7fba7bc83640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba74160770 0x7fba7415c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.826+0000 7fba7bc83640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 0x7fba7415a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.826+0000 7fba7bc83640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fba74145b90 con 0x7fba741637e0 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.826+0000 7fba7bc83640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fba74145a10 con 0x7fba74160770 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.826+0000 7fba7bc83640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fba74169460 con 0x7fba74160390 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.826+0000 7fba7a220640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 0x7fba7415a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7a220640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 0x7fba7415a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44392/0 (socket says 172.21.15.79:44392) 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7a220640 1 -- 172.21.15.79:0/2426203871 learned_addr learned my addr 172.21.15.79:0/2426203871 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba79a1f640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fba74160390 0x7fba74149490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7921e640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba74160770 0x7fba7415c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7a220640 1 -- 172.21.15.79:0/2426203871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fba74160390 msgr2=0x7fba74149490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7a220640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fba74160390 0x7fba74149490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7a220640 1 -- 172.21.15.79:0/2426203871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba74160770 msgr2=0x7fba7415c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:16.831 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7a220640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba74160770 0x7fba7415c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7a220640 1 -- 172.21.15.79:0/2426203871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba7413fc80 con 0x7fba741637e0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7a220640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 0x7fba7415a150 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009f40 tx=0x7fba6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba6a7fc640 1 -- 172.21.15.79:0/2426203871 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c03f020 con 0x7fba741637e0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba6a7fc640 1 -- 172.21.15.79:0/2426203871 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba6c037040 con 0x7fba741637e0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba7413ff60 con 0x7fba741637e0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba6a7fc640 1 -- 172.21.15.79:0/2426203871 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c036070 con 0x7fba741637e0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.827+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba74140520 con 0x7fba741637e0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.828+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba740506c0 con 0x7fba741637e0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.829+0000 7fba6a7fc640 1 -- 172.21.15.79:0/2426203871 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7fba6c042020 con 0x7fba741637e0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.829+0000 7fba6a7fc640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fba5806ca10 0x7fba5806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.829+0000 7fba6a7fc640 1 -- 172.21.15.79:0/2426203871 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fba6c0ac280 con 0x7fba741637e0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.829+0000 7fba79a1f640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fba5806ca10 0x7fba5806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:16.832 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.830+0000 7fba79a1f640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fba5806ca10 0x7fba5806eed0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba700079a0 tx=0x7fba70008040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:16.834 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:16.834+0000 7fba6a7fc640 1 -- 172.21.15.79:0/2426203871 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fba6c049030 con 0x7fba741637e0 2024-04-26T19:52:17.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.007+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fba740b2500 con 0x7fba5806ca10 2024-04-26T19:52:17.008 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.007+0000 7fba6a7fc640 1 -- 172.21.15.79:0/2426203871 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fba740b2500 con 0x7fba5806ca10 2024-04-26T19:52:17.008 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 3 to deep-scrub 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fba5806ca10 msgr2=0x7fba5806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fba5806ca10 0x7fba5806eed0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba700079a0 tx=0x7fba70008040 comp rx=0 tx=0).stop 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 msgr2=0x7fba7415a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 0x7fba7415a150 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009f40 tx=0x7fba6c004a40 comp rx=0 tx=0).stop 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 shutdown_connections 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fba5806ca10 0x7fba5806eed0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fba741637e0 0x7fba7415a150 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fba74160770 0x7fba7415c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 --2- 172.21.15.79:0/2426203871 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fba74160390 0x7fba74149490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 >> 172.21.15.79:0/2426203871 conn(0x7fba740b2720 msgr2=0x7fba741630b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 shutdown_connections 2024-04-26T19:52:17.011 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.009+0000 7fba7bc83640 1 -- 172.21.15.79:0/2426203871 wait complete. 2024-04-26T19:52:17.020 INFO:tasks.ceph:Scrubbing osd.4 2024-04-26T19:52:17.020 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-26T19:52:17.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.120+0000 7f1000112640 1 Processor -- start 2024-04-26T19:52:17.121 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.121+0000 7f1000112640 1 -- start start 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.121+0000 7f1000112640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f0ff80cd720 0x7f0ff806ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.121+0000 7f1000112640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ff80cdc90 con 0x7f0ff80d0cf0 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.121+0000 7f1000112640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ff80cdf70 con 0x7f0ff815b980 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.121+0000 7f1000112640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff80ce250 con 0x7f0ff80cd720 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffd6ad640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f0ff80d0cf0 0x7f0ff806e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:50962/0 (socket says 172.21.15.79:50962) 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffd6ad640 1 -- 172.21.15.79:0/2789497844 learned_addr learned my addr 172.21.15.79:0/2789497844 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffe6af640 1 --2- 172.21.15.79:0/2789497844 >> v2:172.21.15.83:3300/0 conn(0x7f0ff80cd720 0x7f0ff806ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4205265896 0 0) 0x7f0ff80cdc90 con 0x7f0ff80d0cf0 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fe4003520 con 0x7f0ff80d0cf0 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 <== mon.1 v1:172.21.15.83:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1058529041 0 0) 0x7f0ff80cdf70 con 0x7f0ff815b980 2024-04-26T19:52:17.126 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 --> v1:172.21.15.83:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ff80cdc90 con 0x7f0ff815b980 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3510117497 0 0) 0x7f0fe4003520 con 0x7f0ff80d0cf0 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 >> v2:172.21.15.83:3300/0 conn(0x7f0ff80cd720 msgr2=0x7f0ff806ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 --2- 172.21.15.79:0/2789497844 >> v2:172.21.15.83:3300/0 conn(0x7f0ff80cd720 0x7f0ff806ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 >> v1:172.21.15.83:6789/0 conn(0x7f0ff815b980 legacy=0x7f0ff80d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff80ce530 con 0x7f0ff80d0cf0 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f0ff4001670 con 0x7f0ff80d0cf0 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 >> v1:172.21.15.79:6789/0 conn(0x7f0ff80d0cf0 legacy=0x7f0ff806e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe4003b90 0x7f0fe4003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fe4004800 0x7f0fe4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe4005210 0x7f0fe400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff80ce250 con 0x7f0fe4005210 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe400c140 con 0x7f0fe4003b90 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe400c340 con 0x7f0fe4004800 2024-04-26T19:52:17.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffe6af640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe4005210 0x7f0fe400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffdeae640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe4003b90 0x7f0fe4003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.122+0000 7f0ffd6ad640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fe4004800 0x7f0fe4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffe6af640 1 -- 172.21.15.79:0/2789497844 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fe4004800 msgr2=0x7f0fe4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffe6af640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0fe4004800 0x7f0fe4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffe6af640 1 -- 172.21.15.79:0/2789497844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe4003b90 msgr2=0x7f0fe4003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffe6af640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe4003b90 0x7f0fe4003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffe6af640 1 -- 172.21.15.79:0/2789497844 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe4003520 con 0x7f0fe4005210 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffe6af640 1 -- 172.21.15.79:0/2789497844 reap_dead start 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffe6af640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe4005210 0x7f0fe400b9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009fc0 tx=0x7f0fe4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe5cb72af8bcc56 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0035020 con 0x7f0fe4005210 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff0003bd0 con 0x7f0fe4005210 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.123+0000 7f0ffceac640 1 -- 172.21.15.79:0/2789497844 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0009c10 con 0x7f0fe4005210 2024-04-26T19:52:17.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.124+0000 7f1000112640 1 -- 172.21.15.79:0/2789497844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe4005210 msgr2=0x7f0fe400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.124+0000 7f1000112640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe4005210 0x7f0fe400b9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0009fc0 tx=0x7f0fe4005830 comp rx=0 tx=0).stop 2024-04-26T19:52:17.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.124+0000 7f1000112640 1 -- 172.21.15.79:0/2789497844 shutdown_connections 2024-04-26T19:52:17.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.124+0000 7f1000112640 1 --2- 172.21.15.79:0/2789497844 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0fe4005210 0x7f0fe400b9c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.124+0000 7f1000112640 1 -- 172.21.15.79:0/2789497844 >> 172.21.15.79:0/2789497844 conn(0x7f0ff8068840 msgr2=0x7f0ff80d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.124+0000 7f1000112640 1 -- 172.21.15.79:0/2789497844 shutdown_connections 2024-04-26T19:52:17.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.124+0000 7f1000112640 1 -- 172.21.15.79:0/2789497844 wait complete. 2024-04-26T19:52:17.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.125+0000 7f1000112640 1 Processor -- start 2024-04-26T19:52:17.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.125+0000 7f1000112640 1 -- start start 2024-04-26T19:52:17.132 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f1000112640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe4005210 0x7f0ff8145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f1000112640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0ff80b1e80 0x7f0ff81473a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f1000112640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff80b2300 0x7f0ff81478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f1000112640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff815f530 con 0x7f0ff80b2300 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f1000112640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff80cd200 con 0x7f0fe4005210 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f1000112640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff815f6b0 con 0x7f0ff80b1e80 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffe6af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff80b2300 0x7f0ff81478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffe6af640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff80b2300 0x7f0ff81478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44408/0 (socket says 172.21.15.79:44408) 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffe6af640 1 -- 172.21.15.79:0/3579339623 learned_addr learned my addr 172.21.15.79:0/3579339623 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffd6ad640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0ff80b1e80 0x7f0ff81473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffdeae640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe4005210 0x7f0ff8145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffe6af640 1 -- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0ff80b1e80 msgr2=0x7f0ff81473a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffe6af640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f0ff80b1e80 0x7f0ff81473a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.133 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffe6af640 1 -- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe4005210 msgr2=0x7f0ff8145c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffe6af640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe4005210 0x7f0ff8145c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffe6af640 1 -- 172.21.15.79:0/3579339623 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff80b10c0 con 0x7f0ff80b2300 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffdeae640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f0fe4005210 0x7f0ff8145c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.126+0000 7f0ffe6af640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff80b2300 0x7f0ff81478e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000e200 tx=0x7f0ff0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.127+0000 7f0fee7fc640 1 -- 172.21.15.79:0/3579339623 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0035030 con 0x7f0ff80b2300 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.127+0000 7f0fee7fc640 1 -- 172.21.15.79:0/3579339623 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ff0043070 con 0x7f0ff80b2300 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.127+0000 7f0fee7fc640 1 -- 172.21.15.79:0/3579339623 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff002d070 con 0x7f0ff80b2300 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.127+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff80b13a0 con 0x7f0ff80b2300 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.127+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff80b1960 con 0x7f0ff80b2300 2024-04-26T19:52:17.134 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.127+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ff80506c0 con 0x7f0ff80b2300 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.128+0000 7f0fee7fc640 1 -- 172.21.15.79:0/3579339623 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f0ff0005d10 con 0x7f0ff80b2300 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.128+0000 7f0fee7fc640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f0fdc06ca10 0x7f0fdc06eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.128+0000 7f0fee7fc640 1 -- 172.21.15.79:0/3579339623 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f0ff00acc60 con 0x7f0ff80b2300 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.128+0000 7f0ffdeae640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f0fdc06ca10 0x7f0fdc06eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.129+0000 7f0fee7fc640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7f0fdc073400 0x7f0fdc075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.129+0000 7f0fee7fc640 1 -- 172.21.15.79:0/3579339623 --> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ff000df90 con 0x7f0fdc073400 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.129+0000 7f0fee7fc640 1 -- 172.21.15.79:0/3579339623 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f0ff00b12a0 con 0x7f0ff80b2300 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.129+0000 7f0ffd6ad640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7f0fdc073400 0x7f0fdc075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.129+0000 7f0ffdeae640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f0fdc06ca10 0x7f0fdc06eed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80055a0 tx=0x7f0fe8009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.129+0000 7f0ffd6ad640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7f0fdc073400 0x7f0fdc075840 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.135 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.130+0000 7f0fee7fc640 1 -- 172.21.15.79:0/3579339623 <== osd.4 v2:172.21.15.83:6808/3328215207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0ff000df90 con 0x7f0fdc073400 2024-04-26T19:52:17.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.143+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 --> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0ff80d1670 con 0x7f0fdc073400 2024-04-26T19:52:17.143 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.143+0000 7f0fee7fc640 1 -- 172.21.15.79:0/3579339623 <== osd.4 v2:172.21.15.83:6808/3328215207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0ff80d1670 con 0x7f0fdc073400 2024-04-26T19:52:17.144 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:52:17.144 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T19:52:17.144 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.143+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7f0fdc073400 msgr2=0x7f0fdc075840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.143+0000 7f1000112640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.83:6808/3328215207,v1:172.21.15.83:6809/3328215207] conn(0x7f0fdc073400 0x7f0fdc075840 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.143+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f0fdc06ca10 msgr2=0x7f0fdc06eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.143+0000 7f1000112640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f0fdc06ca10 0x7f0fdc06eed0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80055a0 tx=0x7f0fe8009040 comp rx=0 tx=0).stop 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.143+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff80b2300 msgr2=0x7f0ff81478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.143+0000 7f1000112640 1 --2- 172.21.15.79:0/3579339623 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f0ff80b2300 0x7f0ff81478e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000e200 tx=0x7f0ff0009750 comp rx=0 tx=0).stop 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.143+0000 7f0ffe6af640 1 -- 172.21.15.79:0/3579339623 reap_dead start 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.144+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 shutdown_connections 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.144+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 >> 172.21.15.79:0/3579339623 conn(0x7f0ff8068840 msgr2=0x7f0ff814c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.145 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.144+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 shutdown_connections 2024-04-26T19:52:17.146 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.144+0000 7f1000112640 1 -- 172.21.15.79:0/3579339623 wait complete. 2024-04-26T19:52:17.154 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-26T19:52:17.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.253+0000 7f1e7290a640 1 Processor -- start 2024-04-26T19:52:17.253 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.253+0000 7f1e7290a640 1 -- start start 2024-04-26T19:52:17.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.253+0000 7f1e7290a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c163060 0x7f1e6c1604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.253+0000 7f1e7290a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1e6c0708a0 0x7f1e6c0d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.253+0000 7f1e7290a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6c1634f0 con 0x7f1e6c0704c0 2024-04-26T19:52:17.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.253+0000 7f1e7290a640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6c142840 con 0x7f1e6c0708a0 2024-04-26T19:52:17.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.253+0000 7f1e7290a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c1637d0 con 0x7f1e6c163060 2024-04-26T19:52:17.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.253+0000 7f1e70ea7640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1e6c0708a0 0x7f1e6c0d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.254+0000 7f1e70ea7640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f1e6c0708a0 0x7f1e6c0d1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44416/0 (socket says 172.21.15.79:44416) 2024-04-26T19:52:17.255 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.254+0000 7f1e70ea7640 1 -- 172.21.15.79:0/2824821569 learned_addr learned my addr 172.21.15.79:0/2824821569 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:17.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.254+0000 7f1e6b7fe640 1 --2- 172.21.15.79:0/2824821569 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c163060 0x7f1e6c1604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.254+0000 7f1e70ea7640 1 -- 172.21.15.79:0/2824821569 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c163060 msgr2=0x7f1e6c1604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.254+0000 7f1e70ea7640 1 --2- 172.21.15.79:0/2824821569 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c163060 0x7f1e6c1604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.254+0000 7f1e70ea7640 1 -- 172.21.15.79:0/2824821569 >> v1:172.21.15.83:6789/0 conn(0x7f1e6c0704c0 legacy=0x7f1e6c142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.254+0000 7f1e70ea7640 1 -- 172.21.15.79:0/2824821569 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c163ab0 con 0x7f1e6c0708a0 2024-04-26T19:52:17.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.254+0000 7f1e70ea7640 1 --2- 172.21.15.79:0/2824821569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 0x7f1e6c0d1430 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009e50 tx=0x7f1e6402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e9f0f935f26f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e6affd640 1 -- 172.21.15.79:0/2824821569 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e64036070 con 0x7f1e6c0708a0 2024-04-26T19:52:17.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e6affd640 1 -- 172.21.15.79:0/2824821569 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e64031600 con 0x7f1e6c0708a0 2024-04-26T19:52:17.256 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e6affd640 1 -- 172.21.15.79:0/2824821569 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e640319d0 con 0x7f1e6c0708a0 2024-04-26T19:52:17.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e7290a640 1 -- 172.21.15.79:0/2824821569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 msgr2=0x7f1e6c0d1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e7290a640 1 --2- 172.21.15.79:0/2824821569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 0x7f1e6c0d1430 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1e64009e50 tx=0x7f1e6402a710 comp rx=0 tx=0).stop 2024-04-26T19:52:17.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e7290a640 1 -- 172.21.15.79:0/2824821569 shutdown_connections 2024-04-26T19:52:17.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e7290a640 1 --2- 172.21.15.79:0/2824821569 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 0x7f1e6c0d1430 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.257 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e7290a640 1 --2- 172.21.15.79:0/2824821569 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c163060 0x7f1e6c1604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e7290a640 1 -- 172.21.15.79:0/2824821569 >> 172.21.15.79:0/2824821569 conn(0x7f1e6c073820 msgr2=0x7f1e6c0d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e7290a640 1 -- 172.21.15.79:0/2824821569 shutdown_connections 2024-04-26T19:52:17.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.255+0000 7f1e7290a640 1 -- 172.21.15.79:0/2824821569 wait complete. 2024-04-26T19:52:17.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.256+0000 7f1e7290a640 1 Processor -- start 2024-04-26T19:52:17.258 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e7290a640 1 -- start start 2024-04-26T19:52:17.260 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e7290a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c0704c0 0x7f1e6c141a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e7290a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 0x7f1e6c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e7290a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e6c1f5e50 0x7f1e6c158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e7290a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c0d1970 con 0x7f1e6c0708a0 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e7290a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c06fee0 con 0x7f1e6c1f5e50 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e7290a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c0d1af0 con 0x7f1e6c0704c0 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e6b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 0x7f1e6c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e6b7fe640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 0x7f1e6c156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:44418/0 (socket says 172.21.15.79:44418) 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e6b7fe640 1 -- 172.21.15.79:0/593808527 learned_addr learned my addr 172.21.15.79:0/593808527 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e6bfff640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c0704c0 0x7f1e6c141a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.257+0000 7f1e70ea7640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e6c1f5e50 0x7f1e6c158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e6b7fe640 1 -- 172.21.15.79:0/593808527 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c0704c0 msgr2=0x7f1e6c141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e6b7fe640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c0704c0 0x7f1e6c141a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.261 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e6b7fe640 1 -- 172.21.15.79:0/593808527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e6c1f5e50 msgr2=0x7f1e6c158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e6b7fe640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e6c1f5e50 0x7f1e6c158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e6b7fe640 1 -- 172.21.15.79:0/593808527 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c1573e0 con 0x7f1e6c0708a0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e6b7fe640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 0x7f1e6c156b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00d5f0 tx=0x7f1e5c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e68ff9640 1 -- 172.21.15.79:0/593808527 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0125f0 con 0x7f1e6c0708a0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e68ff9640 1 -- 172.21.15.79:0/593808527 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c019070 con 0x7f1e6c0708a0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e68ff9640 1 -- 172.21.15.79:0/593808527 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0129d0 con 0x7f1e6c0708a0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c1576c0 con 0x7f1e6c0708a0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.258+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c157c80 con 0x7f1e6c0708a0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.259+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c158080 con 0x7f1e6c0708a0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.259+0000 7f1e68ff9640 1 -- 172.21.15.79:0/593808527 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f1e5c012b70 con 0x7f1e6c0708a0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.260+0000 7f1e68ff9640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1e5806ca10 0x7f1e5806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.260+0000 7f1e68ff9640 1 -- 172.21.15.79:0/593808527 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f1e5c08c0d0 con 0x7f1e6c0708a0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.260+0000 7f1e6bfff640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1e5806ca10 0x7f1e5806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.262 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.260+0000 7f1e6bfff640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1e5806ca10 0x7f1e5806eed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0ce440 tx=0x7f1e60009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.265 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.264+0000 7f1e68ff9640 1 -- 172.21.15.79:0/593808527 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f1e5c059cb0 con 0x7f1e6c0708a0 2024-04-26T19:52:17.441 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.441+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f1e6c0506c0 con 0x7f1e5806ca10 2024-04-26T19:52:17.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.441+0000 7f1e68ff9640 1 -- 172.21.15.79:0/593808527 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1e6c0506c0 con 0x7f1e5806ca10 2024-04-26T19:52:17.442 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 4 to deep-scrub 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1e5806ca10 msgr2=0x7f1e5806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1e5806ca10 0x7f1e5806eed0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1e6c0ce440 tx=0x7f1e60009040 comp rx=0 tx=0).stop 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 msgr2=0x7f1e6c156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 0x7f1e6c156b00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00d5f0 tx=0x7f1e5c00dab0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 shutdown_connections 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f1e5806ca10 0x7f1e5806eed0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f1e6c1f5e50 0x7f1e6c158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f1e6c0708a0 0x7f1e6c156b00 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 --2- 172.21.15.79:0/593808527 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f1e6c0704c0 0x7f1e6c141a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 >> 172.21.15.79:0/593808527 conn(0x7f1e6c073820 msgr2=0x7f1e6c0c6390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 shutdown_connections 2024-04-26T19:52:17.444 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.443+0000 7f1e7290a640 1 -- 172.21.15.79:0/593808527 wait complete. 2024-04-26T19:52:17.455 INFO:tasks.ceph:Scrubbing osd.5 2024-04-26T19:52:17.455 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-26T19:52:17.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.555+0000 7f632482a640 1 Processor -- start 2024-04-26T19:52:17.556 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.555+0000 7f632482a640 1 -- start start 2024-04-26T19:52:17.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f632482a640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f632015b980 0x7f63200d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f632482a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f632006f370 0x7f63200d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f632482a640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63200cdc90 con 0x7f63200cd720 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f632482a640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f63200cdf70 con 0x7f632015b980 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f632482a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f63200ce250 con 0x7f632006f370 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f6317fff640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f63200cd720 0x7f63200cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33690/0 (socket says 172.21.15.79:33690) 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f6317fff640 1 -- 172.21.15.79:0/1927623305 learned_addr learned my addr 172.21.15.79:0/1927623305 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f631e59c640 1 --2- 172.21.15.79:0/1927623305 >> v2:172.21.15.83:3300/0 conn(0x7f632015b980 0x7f63200d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f631ed9d640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f632006f370 0x7f63200d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1217195025 0 0) 0x7f63200cdc90 con 0x7f63200cd720 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f630c003520 con 0x7f63200cd720 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 211906373 0 0) 0x7f630c003520 con 0x7f63200cd720 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f632006f370 msgr2=0x7f63200d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f631dd9b640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f632006f370 0x7f63200d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 >> v2:172.21.15.83:3300/0 conn(0x7f632015b980 msgr2=0x7f63200d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.556+0000 7f631dd9b640 1 --2- 172.21.15.79:0/1927623305 >> v2:172.21.15.83:3300/0 conn(0x7f632015b980 0x7f63200d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63200ce530 con 0x7f63200cd720 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7f6308001670 con 0x7f63200cd720 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 >> v1:172.21.15.79:6789/0 conn(0x7f63200cd720 legacy=0x7f63200cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631dd9b640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f632006f370 0x7f630c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631dd9b640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f630c003f30 0x7f630c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631dd9b640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f630c004d40 0x7f630c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f63200cdf70 con 0x7f630c004d40 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63200ce250 con 0x7f630c003f30 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f630c00dc20 con 0x7f632006f370 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631ed9d640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f630c004d40 0x7f630c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631e59c640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f632006f370 0x7f630c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f6317fff640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f630c003f30 0x7f630c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631ed9d640 1 -- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f632006f370 msgr2=0x7f630c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631ed9d640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f632006f370 0x7f630c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631ed9d640 1 -- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f630c003f30 msgr2=0x7f630c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631ed9d640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f630c003f30 0x7f630c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f631ed9d640 1 -- 172.21.15.79:0/1927623305 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c003520 con 0x7f630c004d40 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.557+0000 7f6317fff640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f630c003f30 0x7f630c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f631ed9d640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f630c004d40 0x7f630c0051c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6318009fc0 tx=0x7f630c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d468dd5d8f9a3fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318009c10 con 0x7f630c004d40 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6318036070 con 0x7f630c004d40 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f631dd9b640 1 -- 172.21.15.79:0/1927623305 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318031840 con 0x7f630c004d40 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f632482a640 1 -- 172.21.15.79:0/1927623305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f630c004d40 msgr2=0x7f630c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f632482a640 1 --2- 172.21.15.79:0/1927623305 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f630c004d40 0x7f630c0051c0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6318009fc0 tx=0x7f630c005830 comp rx=0 tx=0).stop 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f631ed9d640 1 -- 172.21.15.79:0/1927623305 reap_dead start 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f632482a640 1 -- 172.21.15.79:0/1927623305 shutdown_connections 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f632482a640 1 -- 172.21.15.79:0/1927623305 >> 172.21.15.79:0/1927623305 conn(0x7f6320068840 msgr2=0x7f63200d1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f632482a640 1 -- 172.21.15.79:0/1927623305 shutdown_connections 2024-04-26T19:52:17.563 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.558+0000 7f632482a640 1 -- 172.21.15.79:0/1927623305 wait complete. 2024-04-26T19:52:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f632482a640 1 Processor -- start 2024-04-26T19:52:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f632482a640 1 -- start start 2024-04-26T19:52:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f632482a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6320141420 0x7f632015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f632482a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6320141800 0x7f63201e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f632482a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63201e6b30 0x7f632015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f632482a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f632015f530 con 0x7f63201e6b30 2024-04-26T19:52:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f632482a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f63200cd200 con 0x7f6320141800 2024-04-26T19:52:17.566 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f632482a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f632015f6b0 con 0x7f6320141420 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f631ed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63201e6b30 0x7f632015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f631ed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63201e6b30 0x7f632015bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40200/0 (socket says 172.21.15.79:40200) 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.560+0000 7f631ed9d640 1 -- 172.21.15.79:0/4153167148 learned_addr learned my addr 172.21.15.79:0/4153167148 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f6317fff640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6320141800 0x7f63201e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f631e59c640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6320141420 0x7f632015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f631ed9d640 1 -- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6320141420 msgr2=0x7f632015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f631ed9d640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f6320141420 0x7f632015a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f631ed9d640 1 -- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6320141800 msgr2=0x7f63201e65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f631ed9d640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6320141800 0x7f63201e65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f631ed9d640 1 -- 172.21.15.79:0/4153167148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63201f4360 con 0x7f63201e6b30 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f6317fff640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f6320141800 0x7f63201e65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f631ed9d640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63201e6b30 0x7f632015bf70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6318033040 tx=0x7f6318009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f6316ffd640 1 -- 172.21.15.79:0/4153167148 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318009c10 con 0x7f63201e6b30 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f6316ffd640 1 -- 172.21.15.79:0/4153167148 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6318031d50 con 0x7f63201e6b30 2024-04-26T19:52:17.567 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63201f4640 con 0x7f63201e6b30 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f6316ffd640 1 -- 172.21.15.79:0/4153167148 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6318045650 con 0x7f63201e6b30 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.561+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63201f4c00 con 0x7f63201e6b30 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.562+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63200506c0 con 0x7f63201e6b30 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.562+0000 7f6316ffd640 1 -- 172.21.15.79:0/4153167148 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f6318005d10 con 0x7f63201e6b30 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.563+0000 7f6316ffd640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f630006ca40 0x7f630006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.563+0000 7f6316ffd640 1 -- 172.21.15.79:0/4153167148 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f63180acae0 con 0x7f63201e6b30 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.563+0000 7f631e59c640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f630006ca40 0x7f630006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.563+0000 7f6316ffd640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f6300073430 0x7f6300075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.563+0000 7f6316ffd640 1 -- 172.21.15.79:0/4153167148 --> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f631800df90 con 0x7f6300073430 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.563+0000 7f6316ffd640 1 -- 172.21.15.79:0/4153167148 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f63180b12c0 con 0x7f63201e6b30 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.563+0000 7f6317fff640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f6300073430 0x7f6300075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.563+0000 7f631e59c640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f630006ca40 0x7f630006ef00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6310009b80 tx=0x7f6310008040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.564+0000 7f6317fff640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f6300073430 0x7f6300075870 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.568 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.564+0000 7f6316ffd640 1 -- 172.21.15.79:0/4153167148 <== osd.5 v2:172.21.15.83:6824/3851137838 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f631800df90 con 0x7f6300073430 2024-04-26T19:52:17.578 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.577+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 --> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f63200b4520 con 0x7f6300073430 2024-04-26T19:52:17.579 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f6316ffd640 1 -- 172.21.15.79:0/4153167148 <== osd.5 v2:172.21.15.83:6824/3851137838 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f63200b4520 con 0x7f6300073430 2024-04-26T19:52:17.579 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:52:17.579 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T19:52:17.579 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:52:17.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f6300073430 msgr2=0x7f6300075870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.83:6824/3851137838,v1:172.21.15.83:6825/3851137838] conn(0x7f6300073430 0x7f6300075870 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f630006ca40 msgr2=0x7f630006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f630006ca40 0x7f630006ef00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6310009b80 tx=0x7f6310008040 comp rx=0 tx=0).stop 2024-04-26T19:52:17.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63201e6b30 msgr2=0x7f632015bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 --2- 172.21.15.79:0/4153167148 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f63201e6b30 0x7f632015bf70 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6318033040 tx=0x7f6318009750 comp rx=0 tx=0).stop 2024-04-26T19:52:17.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f631ed9d640 1 -- 172.21.15.79:0/4153167148 reap_dead start 2024-04-26T19:52:17.580 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 shutdown_connections 2024-04-26T19:52:17.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 >> 172.21.15.79:0/4153167148 conn(0x7f6320068840 msgr2=0x7f6320073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 shutdown_connections 2024-04-26T19:52:17.581 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.578+0000 7f632482a640 1 -- 172.21.15.79:0/4153167148 wait complete. 2024-04-26T19:52:17.589 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-26T19:52:17.686 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.686+0000 7fd35312c640 1 Processor -- start 2024-04-26T19:52:17.687 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.686+0000 7fd35312c640 1 -- start start 2024-04-26T19:52:17.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd35312c640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fd34c0cd9c0 0x7fd34c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd35312c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd34c0735a0 0x7fd34c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd35312c640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd34c0ce1a0 con 0x7fd34c156670 2024-04-26T19:52:17.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd35312c640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fd34c073110 con 0x7fd34c0735a0 2024-04-26T19:52:17.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd35312c640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fd34c0cdec0 con 0x7fd34c0cd9c0 2024-04-26T19:52:17.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd3516c9640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd34c0735a0 0x7fd34c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd3516c9640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fd34c0735a0 0x7fd34c0b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40210/0 (socket says 172.21.15.79:40210) 2024-04-26T19:52:17.688 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd3516c9640 1 -- 172.21.15.79:0/12767001 learned_addr learned my addr 172.21.15.79:0/12767001 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:17.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd3516c9640 1 -- 172.21.15.79:0/12767001 >> v2:172.21.15.83:3300/0 conn(0x7fd34c0cd9c0 msgr2=0x7fd34c0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd3516c9640 1 --2- 172.21.15.79:0/12767001 >> v2:172.21.15.83:3300/0 conn(0x7fd34c0cd9c0 0x7fd34c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd3516c9640 1 -- 172.21.15.79:0/12767001 >> v1:172.21.15.83:6789/0 conn(0x7fd34c156670 legacy=0x7fd34c15d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.689 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.687+0000 7fd3516c9640 1 -- 172.21.15.79:0/12767001 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c072ac0 con 0x7fd34c0735a0 2024-04-26T19:52:17.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.688+0000 7fd3516c9640 1 --2- 172.21.15.79:0/12767001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c0735a0 0x7fd34c0b4320 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd34c074170 tx=0x7fd34402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961ede16b0945d2e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.690 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.688+0000 7fd34b7fe640 1 -- 172.21.15.79:0/12767001 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344036070 con 0x7fd34c0735a0 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.688+0000 7fd34b7fe640 1 -- 172.21.15.79:0/12767001 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd344031600 con 0x7fd34c0735a0 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.688+0000 7fd34b7fe640 1 -- 172.21.15.79:0/12767001 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3440319d0 con 0x7fd34c0735a0 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.688+0000 7fd35312c640 1 -- 172.21.15.79:0/12767001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c0735a0 msgr2=0x7fd34c0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.688+0000 7fd35312c640 1 --2- 172.21.15.79:0/12767001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c0735a0 0x7fd34c0b4320 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd34c074170 tx=0x7fd34402a6f0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.689+0000 7fd35312c640 1 -- 172.21.15.79:0/12767001 shutdown_connections 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.689+0000 7fd35312c640 1 --2- 172.21.15.79:0/12767001 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c0735a0 0x7fd34c0b4320 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.689+0000 7fd35312c640 1 --2- 172.21.15.79:0/12767001 >> v2:172.21.15.83:3300/0 conn(0x7fd34c0cd9c0 0x7fd34c0b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.689+0000 7fd35312c640 1 -- 172.21.15.79:0/12767001 >> 172.21.15.79:0/12767001 conn(0x7fd34c0b39d0 msgr2=0x7fd34c06feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.689+0000 7fd35312c640 1 -- 172.21.15.79:0/12767001 shutdown_connections 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.689+0000 7fd35312c640 1 -- 172.21.15.79:0/12767001 wait complete. 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.690+0000 7fd35312c640 1 Processor -- start 2024-04-26T19:52:17.691 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.690+0000 7fd35312c640 1 -- start start 2024-04-26T19:52:17.694 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd35312c640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd34c0735a0 0x7fd34c1409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd35312c640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd34c0cd9c0 0x7fd34c13f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd35312c640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c156670 0x7fd34c13e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd35312c640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c156bd0 con 0x7fd34c156670 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd35312c640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fd34c156a50 con 0x7fd34c0cd9c0 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd35312c640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fd34c0cf650 con 0x7fd34c0735a0 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3516c9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c156670 0x7fd34c13e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3516c9640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c156670 0x7fd34c13e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40214/0 (socket says 172.21.15.79:40214) 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3516c9640 1 -- 172.21.15.79:0/270674647 learned_addr learned my addr 172.21.15.79:0/270674647 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3516c9640 1 -- 172.21.15.79:0/270674647 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd34c0735a0 msgr2=0x7fd34c1409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3516c9640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd34c0735a0 0x7fd34c1409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3516c9640 1 -- 172.21.15.79:0/270674647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd34c0cd9c0 msgr2=0x7fd34c13f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3516c9640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd34c0cd9c0 0x7fd34c13f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3516c9640 1 -- 172.21.15.79:0/270674647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34c140f20 con 0x7fd34c156670 2024-04-26T19:52:17.695 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3516c9640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c156670 0x7fd34c13e830 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd344034040 tx=0x7fd344004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3497fa640 1 -- 172.21.15.79:0/270674647 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344041020 con 0x7fd34c156670 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3497fa640 1 -- 172.21.15.79:0/270674647 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd344007780 con 0x7fd34c156670 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34c13f940 con 0x7fd34c156670 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd3497fa640 1 -- 172.21.15.79:0/270674647 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344036040 con 0x7fd34c156670 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.691+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd34c13ff00 con 0x7fd34c156670 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.692+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34c15bc90 con 0x7fd34c156670 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.693+0000 7fd3497fa640 1 -- 172.21.15.79:0/270674647 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7fd344044020 con 0x7fd34c156670 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.693+0000 7fd3497fa640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fd33006ca40 0x7fd33006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.693+0000 7fd3497fa640 1 -- 172.21.15.79:0/270674647 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fd3440accd0 con 0x7fd34c156670 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.693+0000 7fd350ec8640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fd33006ca40 0x7fd33006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.696 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.694+0000 7fd350ec8640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fd33006ca40 0x7fd33006ef00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd340009650 tx=0x7fd340006ba0 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.698 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.697+0000 7fd3497fa640 1 -- 172.21.15.79:0/270674647 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fd34c15bc90 con 0x7fd34c156670 2024-04-26T19:52:17.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.872+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fd34c0506c0 con 0x7fd33006ca40 2024-04-26T19:52:17.873 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.873+0000 7fd3497fa640 1 -- 172.21.15.79:0/270674647 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd34c0506c0 con 0x7fd33006ca40 2024-04-26T19:52:17.873 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 5 to deep-scrub 2024-04-26T19:52:17.875 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fd33006ca40 msgr2=0x7fd33006ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fd33006ca40 0x7fd33006ef00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd340009650 tx=0x7fd340006ba0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c156670 msgr2=0x7fd34c13e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c156670 0x7fd34c13e830 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd344034040 tx=0x7fd344004a40 comp rx=0 tx=0).stop 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 shutdown_connections 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fd33006ca40 0x7fd33006ef00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fd34c156670 0x7fd34c13e830 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fd34c0cd9c0 0x7fd34c13f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 --2- 172.21.15.79:0/270674647 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fd34c0735a0 0x7fd34c1409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 >> 172.21.15.79:0/270674647 conn(0x7fd34c0b39d0 msgr2=0x7fd34c0b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 shutdown_connections 2024-04-26T19:52:17.876 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.875+0000 7fd35312c640 1 -- 172.21.15.79:0/270674647 wait complete. 2024-04-26T19:52:17.886 INFO:tasks.ceph:Scrubbing osd.6 2024-04-26T19:52:17.886 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-26T19:52:17.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.986+0000 7faca1a72640 1 Processor -- start 2024-04-26T19:52:17.987 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.987+0000 7faca1a72640 1 -- start start 2024-04-26T19:52:17.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.987+0000 7faca1a72640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7fac9c1453b0 0x7fac9c06f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.987+0000 7faca1a72640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c06e480 con 0x7fac9c06df80 2024-04-26T19:52:17.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.987+0000 7faca1a72640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c06e760 con 0x7fac9c159b30 2024-04-26T19:52:17.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.987+0000 7faca1a72640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c145bb0 con 0x7fac9c1453b0 2024-04-26T19:52:17.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9affd640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fac9c06df80 0x7fac9c144ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33692/0 (socket says 172.21.15.79:33692) 2024-04-26T19:52:17.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9affd640 1 -- 172.21.15.79:0/1792850251 learned_addr learned my addr 172.21.15.79:0/1792850251 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:17.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4251814832 0 0) 0x7fac9c06e480 con 0x7fac9c06df80 2024-04-26T19:52:17.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9bfff640 1 --2- 172.21.15.79:0/1792850251 >> v2:172.21.15.83:3300/0 conn(0x7fac9c1453b0 0x7fac9c06f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.991 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac84003520 con 0x7fac9c06df80 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2844740297 0 0) 0x7fac84003520 con 0x7fac9c06df80 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 >> v2:172.21.15.83:3300/0 conn(0x7fac9c1453b0 msgr2=0x7fac9c06f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 --2- 172.21.15.79:0/1792850251 >> v2:172.21.15.83:3300/0 conn(0x7fac9c1453b0 0x7fac9c06f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 >> v1:172.21.15.83:6789/0 conn(0x7fac9c159b30 legacy=0x7fac9c160560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 --> v1:172.21.15.79:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c06ea40 con 0x7fac9c06df80 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 <== mon.0 v1:172.21.15.79:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 58530943 0 0) 0x7fac90001670 con 0x7fac9c06df80 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 >> v1:172.21.15.79:6789/0 conn(0x7fac9c06df80 legacy=0x7fac9c144ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac840039b0 0x7fac84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac84004800 0x7fac84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac840051c0 0x7fac8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac8400c030 con 0x7fac840051c0 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac8400c1e0 con 0x7fac84004800 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fac8400c3e0 con 0x7fac840039b0 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9bfff640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac840051c0 0x7fac8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.992 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9affd640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac84004800 0x7fac84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9b7fe640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac840039b0 0x7fac84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9bfff640 1 -- 172.21.15.79:0/1792850251 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac840039b0 msgr2=0x7fac84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.988+0000 7fac9bfff640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac840039b0 0x7fac84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.989+0000 7fac9bfff640 1 -- 172.21.15.79:0/1792850251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac84004800 msgr2=0x7fac84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.989+0000 7fac9bfff640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac84004800 0x7fac84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.989+0000 7fac9bfff640 1 -- 172.21.15.79:0/1792850251 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac84003520 con 0x7fac840051c0 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.989+0000 7fac9bfff640 1 -- 172.21.15.79:0/1792850251 reap_dead start 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.989+0000 7fac9bfff640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac840051c0 0x7fac8400b9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fac94009fc0 tx=0x7fac84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13606c5b71527a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.989+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94035020 con 0x7fac840051c0 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.989+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac94003bd0 con 0x7fac840051c0 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.989+0000 7fac9a7fc640 1 -- 172.21.15.79:0/1792850251 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94009c10 con 0x7fac840051c0 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.990+0000 7faca1a72640 1 -- 172.21.15.79:0/1792850251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac840051c0 msgr2=0x7fac8400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.990+0000 7faca1a72640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac840051c0 0x7fac8400b9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fac94009fc0 tx=0x7fac84005830 comp rx=0 tx=0).stop 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.990+0000 7faca1a72640 1 -- 172.21.15.79:0/1792850251 shutdown_connections 2024-04-26T19:52:17.993 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.990+0000 7faca1a72640 1 --2- 172.21.15.79:0/1792850251 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac840051c0 0x7fac8400b9c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.990+0000 7faca1a72640 1 -- 172.21.15.79:0/1792850251 >> 172.21.15.79:0/1792850251 conn(0x7fac9c0685d0 msgr2=0x7fac9c0d55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:17.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.990+0000 7faca1a72640 1 -- 172.21.15.79:0/1792850251 shutdown_connections 2024-04-26T19:52:17.994 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.990+0000 7faca1a72640 1 -- 172.21.15.79:0/1792850251 wait complete. 2024-04-26T19:52:17.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.991+0000 7faca1a72640 1 Processor -- start 2024-04-26T19:52:17.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.991+0000 7faca1a72640 1 -- start start 2024-04-26T19:52:17.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7faca1a72640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac840051c0 0x7fac9c141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7faca1a72640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac9c0b15a0 0x7fac9c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7faca1a72640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac9c0b1a20 0x7fac9c155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7faca1a72640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c159f10 con 0x7fac9c0b1a20 2024-04-26T19:52:17.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7faca1a72640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c06f090 con 0x7fac840051c0 2024-04-26T19:52:17.997 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7faca1a72640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c06f520 con 0x7fac9c0b15a0 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac9c0b1a20 0x7fac9c155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9bfff640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac9c0b1a20 0x7fac9c155d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40228/0 (socket says 172.21.15.79:40228) 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9bfff640 1 -- 172.21.15.79:0/2620405429 learned_addr learned my addr 172.21.15.79:0/2620405429 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9b7fe640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac840051c0 0x7fac9c141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9bfff640 1 -- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac9c0b15a0 msgr2=0x7fac9c155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9bfff640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fac9c0b15a0 0x7fac9c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9bfff640 1 -- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac840051c0 msgr2=0x7fac9c141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9bfff640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac840051c0 0x7fac9c141020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9bfff640 1 -- 172.21.15.79:0/2620405429 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c1565c0 con 0x7fac9c0b1a20 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9b7fe640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fac840051c0 0x7fac9c141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.992+0000 7fac9bfff640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac9c0b1a20 0x7fac9c155d50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fac9400e230 tx=0x7fac94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.993+0000 7fac83fff640 1 -- 172.21.15.79:0/2620405429 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac94035030 con 0x7fac9c0b1a20 2024-04-26T19:52:17.998 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.993+0000 7fac83fff640 1 -- 172.21.15.79:0/2620405429 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac94043070 con 0x7fac9c0b1a20 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.993+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c1568a0 con 0x7fac9c0b1a20 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.993+0000 7fac83fff640 1 -- 172.21.15.79:0/2620405429 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9402d070 con 0x7fac9c0b1a20 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.993+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c141560 con 0x7fac9c0b1a20 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.994+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fac9c153590 con 0x7fac9c0b1a20 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.994+0000 7fac83fff640 1 -- 172.21.15.79:0/2620405429 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7fac94005d10 con 0x7fac9c0b1a20 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.995+0000 7fac83fff640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fac8806ca40 0x7fac8806ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.995+0000 7fac83fff640 1 -- 172.21.15.79:0/2620405429 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fac940ac950 con 0x7fac9c0b1a20 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.995+0000 7fac9b7fe640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fac8806ca40 0x7fac8806ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.995+0000 7fac83fff640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7fac88073430 0x7fac88075870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.995+0000 7fac83fff640 1 -- 172.21.15.79:0/2620405429 --> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fac88075f40 con 0x7fac88073430 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.995+0000 7fac83fff640 1 -- 172.21.15.79:0/2620405429 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fac940b12c0 con 0x7fac9c0b1a20 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.995+0000 7fac9b7fe640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fac8806ca40 0x7fac8806ef00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac8c005580 tx=0x7fac8c009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.995+0000 7fac9affd640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7fac88073430 0x7fac88075870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.996+0000 7fac9affd640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7fac88073430 0x7fac88075870 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:17.999 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:17.996+0000 7fac83fff640 1 -- 172.21.15.79:0/2620405429 <== osd.6 v2:172.21.15.83:6800/2962195076 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fac88075f40 con 0x7fac88073430 2024-04-26T19:52:18.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.009+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 --> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fac9c0506c0 con 0x7fac88073430 2024-04-26T19:52:18.010 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7fac83fff640 1 -- 172.21.15.79:0/2620405429 <== osd.6 v2:172.21.15.83:6800/2962195076 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fac9c0506c0 con 0x7fac88073430 2024-04-26T19:52:18.010 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:52:18.011 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T19:52:18.011 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:52:18.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7fac88073430 msgr2=0x7fac88075870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.83:6800/2962195076,v1:172.21.15.83:6801/2962195076] conn(0x7fac88073430 0x7fac88075870 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fac8806ca40 msgr2=0x7fac8806ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fac8806ca40 0x7fac8806ef00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fac8c005580 tx=0x7fac8c009040 comp rx=0 tx=0).stop 2024-04-26T19:52:18.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac9c0b1a20 msgr2=0x7fac9c155d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 --2- 172.21.15.79:0/2620405429 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fac9c0b1a20 0x7fac9c155d50 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fac9400e230 tx=0x7fac94009750 comp rx=0 tx=0).stop 2024-04-26T19:52:18.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7fac9bfff640 1 -- 172.21.15.79:0/2620405429 reap_dead start 2024-04-26T19:52:18.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 shutdown_connections 2024-04-26T19:52:18.012 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 >> 172.21.15.79:0/2620405429 conn(0x7fac9c0685d0 msgr2=0x7fac9c158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:18.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 shutdown_connections 2024-04-26T19:52:18.013 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.010+0000 7faca1a72640 1 -- 172.21.15.79:0/2620405429 wait complete. 2024-04-26T19:52:18.022 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-26T19:52:18.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.119+0000 7f5ba1493640 1 Processor -- start 2024-04-26T19:52:18.120 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.119+0000 7f5ba1493640 1 -- start start 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5ba1493640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f5b9c06fb20 0x7f5b9c0d2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5ba1493640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5b9c06ff00 0x7f5b9c0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5ba1493640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b9c0b1290 con 0x7f5b9c1488c0 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5ba1493640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f5b9c0d3e60 con 0x7f5b9c06ff00 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5ba1493640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f5b9c0d4090 con 0x7f5b9c06fb20 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b9a7fc640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f5b9c06ff00 0x7f5b9c0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b9b7fe640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f5b9c1488c0 0x7f5b9c148d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33700/0 (socket says 172.21.15.79:33700) 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b9b7fe640 1 -- 172.21.15.79:0/3771231501 learned_addr learned my addr 172.21.15.79:0/3771231501 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b9affd640 1 --2- 172.21.15.79:0/3771231501 >> v2:172.21.15.83:3300/0 conn(0x7f5b9c06fb20 0x7f5b9c0d2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b9a7fc640 1 -- 172.21.15.79:0/3771231501 >> v2:172.21.15.83:3300/0 conn(0x7f5b9c06fb20 msgr2=0x7f5b9c0d2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b9a7fc640 1 --2- 172.21.15.79:0/3771231501 >> v2:172.21.15.83:3300/0 conn(0x7f5b9c06fb20 0x7f5b9c0d2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.123 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b9a7fc640 1 -- 172.21.15.79:0/3771231501 >> v1:172.21.15.79:6789/0 conn(0x7f5b9c1488c0 legacy=0x7f5b9c148d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b9a7fc640 1 -- 172.21.15.79:0/3771231501 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c0d4370 con 0x7f5b9c06ff00 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b99ffb640 1 -- 172.21.15.79:0/3771231501 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1191604024 0 0) 0x7f5b9c0b1290 con 0x7f5b9c1488c0 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.120+0000 7f5b9affd640 1 --2- 172.21.15.79:0/3771231501 >> v2:172.21.15.83:3300/0 conn(0x7f5b9c06fb20 0x7f5b9c0d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.121+0000 7f5b9a7fc640 1 --2- 172.21.15.79:0/3771231501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06ff00 0x7f5b9c0d3920 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5b900097d0 tx=0x7f5b9002a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad40274de8d513d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.121+0000 7f5b99ffb640 1 -- 172.21.15.79:0/3771231501 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b90037070 con 0x7f5b9c06ff00 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.121+0000 7f5b99ffb640 1 -- 172.21.15.79:0/3771231501 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b90032440 con 0x7f5b9c06ff00 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.121+0000 7f5b99ffb640 1 -- 172.21.15.79:0/3771231501 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b90032810 con 0x7f5b9c06ff00 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.122+0000 7f5ba1493640 1 -- 172.21.15.79:0/3771231501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06ff00 msgr2=0x7f5b9c0d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.122+0000 7f5ba1493640 1 --2- 172.21.15.79:0/3771231501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06ff00 0x7f5b9c0d3920 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5b900097d0 tx=0x7f5b9002a290 comp rx=0 tx=0).stop 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.122+0000 7f5ba1493640 1 -- 172.21.15.79:0/3771231501 shutdown_connections 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.122+0000 7f5ba1493640 1 --2- 172.21.15.79:0/3771231501 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06ff00 0x7f5b9c0d3920 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.122+0000 7f5ba1493640 1 --2- 172.21.15.79:0/3771231501 >> v2:172.21.15.83:3300/0 conn(0x7f5b9c06fb20 0x7f5b9c0d2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.122+0000 7f5ba1493640 1 -- 172.21.15.79:0/3771231501 >> 172.21.15.79:0/3771231501 conn(0x7f5b9c071c30 msgr2=0x7f5b9c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.122+0000 7f5ba1493640 1 -- 172.21.15.79:0/3771231501 shutdown_connections 2024-04-26T19:52:18.124 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.122+0000 7f5ba1493640 1 -- 172.21.15.79:0/3771231501 wait complete. 2024-04-26T19:52:18.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.123+0000 7f5ba1493640 1 Processor -- start 2024-04-26T19:52:18.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.123+0000 7f5ba1493640 1 -- start start 2024-04-26T19:52:18.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5ba1493640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06fb20 0x7f5b9c0b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5ba1493640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9c06ff00 0x7f5b9c143e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5ba1493640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5b9c1488c0 0x7f5b9c1443d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.127 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5ba1493640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c0d4c10 con 0x7f5b9c06fb20 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5ba1493640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c0d4930 con 0x7f5b9c06ff00 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5ba1493640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f5b9c0aebb0 con 0x7f5b9c1488c0 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b9affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06fb20 0x7f5b9c0b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b9affd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06fb20 0x7f5b9c0b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40250/0 (socket says 172.21.15.79:40250) 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b9affd640 1 -- 172.21.15.79:0/1876800051 learned_addr learned my addr 172.21.15.79:0/1876800051 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b9affd640 1 -- 172.21.15.79:0/1876800051 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5b9c1488c0 msgr2=0x7f5b9c1443d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b9affd640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5b9c1488c0 0x7f5b9c1443d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b9affd640 1 -- 172.21.15.79:0/1876800051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9c06ff00 msgr2=0x7f5b9c143e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b9affd640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9c06ff00 0x7f5b9c143e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b9affd640 1 -- 172.21.15.79:0/1876800051 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c15ffc0 con 0x7f5b9c06fb20 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b9affd640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06fb20 0x7f5b9c0b3190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00cf60 tx=0x7f5b8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b837fe640 1 -- 172.21.15.79:0/1876800051 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c01a070 con 0x7f5b9c06fb20 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b837fe640 1 -- 172.21.15.79:0/1876800051 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b8c00d840 con 0x7f5b9c06fb20 2024-04-26T19:52:18.128 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c1602a0 con 0x7f5b9c06fb20 2024-04-26T19:52:18.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5b837fe640 1 -- 172.21.15.79:0/1876800051 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00dc10 con 0x7f5b9c06fb20 2024-04-26T19:52:18.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.124+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c160860 con 0x7f5b9c06fb20 2024-04-26T19:52:18.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.125+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b9c0506c0 con 0x7f5b9c06fb20 2024-04-26T19:52:18.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.126+0000 7f5b837fe640 1 -- 172.21.15.79:0/1876800051 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f5b8c01d070 con 0x7f5b9c06fb20 2024-04-26T19:52:18.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.126+0000 7f5b837fe640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5b8806ca10 0x7f5b8806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.126+0000 7f5b837fe640 1 -- 172.21.15.79:0/1876800051 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f5b8c08c620 con 0x7f5b9c06fb20 2024-04-26T19:52:18.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.126+0000 7f5b9a7fc640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5b8806ca10 0x7f5b8806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.129 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.127+0000 7f5b9a7fc640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5b8806ca10 0x7f5b8806eed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5b90034000 tx=0x7f5b9002c000 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.131 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.131+0000 7f5b837fe640 1 -- 172.21.15.79:0/1876800051 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f5b8c05a170 con 0x7f5b9c06fb20 2024-04-26T19:52:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.307+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f5b9c0d32d0 con 0x7f5b8806ca10 2024-04-26T19:52:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.307+0000 7f5b837fe640 1 -- 172.21.15.79:0/1876800051 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5b9c0d32d0 con 0x7f5b8806ca10 2024-04-26T19:52:18.308 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 6 to deep-scrub 2024-04-26T19:52:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5b8806ca10 msgr2=0x7f5b8806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5b8806ca10 0x7f5b8806eed0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5b90034000 tx=0x7f5b9002c000 comp rx=0 tx=0).stop 2024-04-26T19:52:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06fb20 msgr2=0x7f5b9c0b3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06fb20 0x7f5b9c0b3190 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c00cf60 tx=0x7f5b8c00cf90 comp rx=0 tx=0).stop 2024-04-26T19:52:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 shutdown_connections 2024-04-26T19:52:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f5b8806ca10 0x7f5b8806eed0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.310 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f5b9c1488c0 0x7f5b9c1443d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5b9c06ff00 0x7f5b9c143e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 --2- 172.21.15.79:0/1876800051 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f5b9c06fb20 0x7f5b9c0b3190 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 >> 172.21.15.79:0/1876800051 conn(0x7f5b9c071c30 msgr2=0x7f5b9c15ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:18.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 shutdown_connections 2024-04-26T19:52:18.311 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.309+0000 7f5ba1493640 1 -- 172.21.15.79:0/1876800051 wait complete. 2024-04-26T19:52:18.321 INFO:tasks.ceph:Scrubbing osd.7 2024-04-26T19:52:18.321 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-26T19:52:18.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.420+0000 7febcfc7b640 1 Processor -- start 2024-04-26T19:52:18.421 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.420+0000 7febcfc7b640 1 -- start start 2024-04-26T19:52:18.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.421+0000 7febcfc7b640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7febc80b3e50 0x7febc8143240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.421+0000 7febcfc7b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7febc80d1620 0x7febc80d1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.421+0000 7febcfc7b640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7febc80aa8c0 con 0x7febc815e0e0 2024-04-26T19:52:18.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.421+0000 7febcfc7b640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7febc8140520 con 0x7febc80b3e50 2024-04-26T19:52:18.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.421+0000 7febcfc7b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7febc80ab940 con 0x7febc80d1620 2024-04-26T19:52:18.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.421+0000 7febcd216640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7febc80b3e50 0x7febc8143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.421+0000 7febcda17640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7febc815e0e0 0x7febc80b3740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33712/0 (socket says 172.21.15.79:33712) 2024-04-26T19:52:18.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.421+0000 7febcda17640 1 -- 172.21.15.79:0/4150539786 learned_addr learned my addr 172.21.15.79:0/4150539786 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:18.423 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.422+0000 7febce218640 1 --2- 172.21.15.79:0/4150539786 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7febc80d1620 0x7febc80d1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.422+0000 7febcd216640 1 -- 172.21.15.79:0/4150539786 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7febc80d1620 msgr2=0x7febc80d1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:18.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.422+0000 7febcd216640 1 --2- 172.21.15.79:0/4150539786 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7febc80d1620 0x7febc80d1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.422+0000 7febcd216640 1 -- 172.21.15.79:0/4150539786 >> v1:172.21.15.79:6789/0 conn(0x7febc815e0e0 legacy=0x7febc80b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.422+0000 7febcd216640 1 -- 172.21.15.79:0/4150539786 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc80aaba0 con 0x7febc80b3e50 2024-04-26T19:52:18.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.422+0000 7febcd216640 1 --2- 172.21.15.79:0/4150539786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80b3e50 0x7febc8143240 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7febbc0097a0 tx=0x7febbc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86129b0bc6ba08e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcca15640 1 -- 172.21.15.79:0/4150539786 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc037070 con 0x7febc80b3e50 2024-04-26T19:52:18.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcca15640 1 -- 172.21.15.79:0/4150539786 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc032440 con 0x7febc80b3e50 2024-04-26T19:52:18.424 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcca15640 1 -- 172.21.15.79:0/4150539786 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc032810 con 0x7febc80b3e50 2024-04-26T19:52:18.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcfc7b640 1 -- 172.21.15.79:0/4150539786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80b3e50 msgr2=0x7febc8143240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcfc7b640 1 --2- 172.21.15.79:0/4150539786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80b3e50 0x7febc8143240 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7febbc0097a0 tx=0x7febbc02a180 comp rx=0 tx=0).stop 2024-04-26T19:52:18.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcfc7b640 1 -- 172.21.15.79:0/4150539786 shutdown_connections 2024-04-26T19:52:18.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcfc7b640 1 --2- 172.21.15.79:0/4150539786 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7febc80d1620 0x7febc80d1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.425 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcfc7b640 1 --2- 172.21.15.79:0/4150539786 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80b3e50 0x7febc8143240 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcfc7b640 1 -- 172.21.15.79:0/4150539786 >> 172.21.15.79:0/4150539786 conn(0x7febc8068390 msgr2=0x7febc80687a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:18.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcfc7b640 1 -- 172.21.15.79:0/4150539786 shutdown_connections 2024-04-26T19:52:18.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.423+0000 7febcfc7b640 1 -- 172.21.15.79:0/4150539786 wait complete. 2024-04-26T19:52:18.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.424+0000 7febcfc7b640 1 Processor -- start 2024-04-26T19:52:18.426 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.424+0000 7febcfc7b640 1 -- start start 2024-04-26T19:52:18.428 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcfc7b640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febc80b3e50 0x7febc8153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcfc7b640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80d1620 0x7febc8132a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcfc7b640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7febc815e0e0 0x7febc8159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcfc7b640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7febc80ab440 con 0x7febc80d1620 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcfc7b640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7febc80ab160 con 0x7febc80b3e50 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcfc7b640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7febc815d9d0 con 0x7febc815e0e0 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcd216640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80d1620 0x7febc8132a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcd216640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80d1620 0x7febc8132a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40266/0 (socket says 172.21.15.79:40266) 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcd216640 1 -- 172.21.15.79:0/2039582105 learned_addr learned my addr 172.21.15.79:0/2039582105 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febce218640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7febc815e0e0 0x7febc8159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcd216640 1 -- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7febc815e0e0 msgr2=0x7febc8159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcda17640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febc80b3e50 0x7febc8153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcd216640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7febc815e0e0 0x7febc8159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcd216640 1 -- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febc80b3e50 msgr2=0x7febc8153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.429 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcd216640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7febc80b3e50 0x7febc8153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.425+0000 7febcd216640 1 -- 172.21.15.79:0/2039582105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc8158de0 con 0x7febc80d1620 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.426+0000 7febcd216640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80d1620 0x7febc8132a30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098c0 tx=0x7febbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.426+0000 7febc67fc640 1 -- 172.21.15.79:0/2039582105 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc037040 con 0x7febc80d1620 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.426+0000 7febc67fc640 1 -- 172.21.15.79:0/2039582105 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7febbc030770 con 0x7febc80d1620 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.426+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc81590c0 con 0x7febc80d1620 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.426+0000 7febc67fc640 1 -- 172.21.15.79:0/2039582105 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc030b40 con 0x7febc80d1620 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.426+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc8159680 con 0x7febc80d1620 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.427+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febc80506c0 con 0x7febc80d1620 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.427+0000 7febc67fc640 1 -- 172.21.15.79:0/2039582105 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7febbc044020 con 0x7febc80d1620 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.428+0000 7febc67fc640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7febb406ca10 0x7febb406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.428+0000 7febc67fc640 1 -- 172.21.15.79:0/2039582105 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7febbc0ac240 con 0x7febc80d1620 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.428+0000 7febcda17640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7febb406ca10 0x7febb406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.428+0000 7febc67fc640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7febb4073400 0x7febb4075840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.430 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.428+0000 7febc67fc640 1 -- 172.21.15.79:0/2039582105 --> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febbc00af30 con 0x7febb4073400 2024-04-26T19:52:18.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.428+0000 7febc67fc640 1 -- 172.21.15.79:0/2039582105 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7febbc0b3050 con 0x7febc80d1620 2024-04-26T19:52:18.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.428+0000 7febcda17640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7febb406ca10 0x7febb406eed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febc00012e0 tx=0x7febc0009040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.428+0000 7febce218640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7febb4073400 0x7febb4075840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.428+0000 7febce218640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7febb4073400 0x7febb4075840 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.431 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.429+0000 7febc67fc640 1 -- 172.21.15.79:0/2039582105 <== osd.7 v2:172.21.15.83:6816/4292059702 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febbc00af30 con 0x7febb4073400 2024-04-26T19:52:18.442 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.442+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 --> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7febc80d1400 con 0x7febb4073400 2024-04-26T19:52:18.443 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febc67fc640 1 -- 172.21.15.79:0/2039582105 <== osd.7 v2:172.21.15.83:6816/4292059702 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7febc80d1400 con 0x7febb4073400 2024-04-26T19:52:18.444 INFO:teuthology.orchestra.run.smithi079.stdout:{ 2024-04-26T19:52:18.444 INFO:teuthology.orchestra.run.smithi079.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T19:52:18.444 INFO:teuthology.orchestra.run.smithi079.stdout:} 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7febb4073400 msgr2=0x7febb4075840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.83:6816/4292059702,v1:172.21.15.83:6817/4292059702] conn(0x7febb4073400 0x7febb4075840 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7febb406ca10 msgr2=0x7febb406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7febb406ca10 0x7febb406eed0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7febc00012e0 tx=0x7febc0009040 comp rx=0 tx=0).stop 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80d1620 msgr2=0x7febc8132a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 --2- 172.21.15.79:0/2039582105 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7febc80d1620 0x7febc8132a30 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098c0 tx=0x7febbc0046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febce218640 1 -- 172.21.15.79:0/2039582105 reap_dead start 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 shutdown_connections 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 >> 172.21.15.79:0/2039582105 conn(0x7febc8068390 msgr2=0x7febc8142da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 shutdown_connections 2024-04-26T19:52:18.445 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.443+0000 7febcfc7b640 1 -- 172.21.15.79:0/2039582105 wait complete. 2024-04-26T19:52:18.454 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-26T19:52:18.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.551+0000 7fdc4bdb4640 1 Processor -- start 2024-04-26T19:52:18.552 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.551+0000 7fdc4bdb4640 1 -- start start 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.552+0000 7fdc4bdb4640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdc440b4180 0x7fdc440d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.552+0000 7fdc4bdb4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 0x7fdc4415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.552+0000 7fdc4bdb4640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc440d27d0 con 0x7fdc44146290 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.552+0000 7fdc4bdb4640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7fdc440ab8a0 con 0x7fdc440b4180 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.552+0000 7fdc4bdb4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fdc440aaf70 con 0x7fdc441469a0 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.552+0000 7fdc4934f640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7fdc440b4180 0x7fdc440d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.552+0000 7fdc49b50640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7fdc44146290 0x7fdc440b3a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33728/0 (socket says 172.21.15.79:33728) 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.552+0000 7fdc49b50640 1 -- 172.21.15.79:0/749299127 learned_addr learned my addr 172.21.15.79:0/749299127 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.553+0000 7fdc4a351640 1 --2- 172.21.15.79:0/749299127 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 0x7fdc4415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.553+0000 7fdc4934f640 1 -- 172.21.15.79:0/749299127 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 msgr2=0x7fdc4415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.553+0000 7fdc4934f640 1 --2- 172.21.15.79:0/749299127 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 0x7fdc4415dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.553+0000 7fdc4934f640 1 -- 172.21.15.79:0/749299127 >> v1:172.21.15.79:6789/0 conn(0x7fdc44146290 legacy=0x7fdc440b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.554 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.553+0000 7fdc48b4e640 1 -- 172.21.15.79:0/749299127 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2424771559 0 0) 0x7fdc440d27d0 con 0x7fdc44146290 2024-04-26T19:52:18.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.553+0000 7fdc4934f640 1 -- 172.21.15.79:0/749299127 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc440ab250 con 0x7fdc440b4180 2024-04-26T19:52:18.555 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.553+0000 7fdc4934f640 1 --2- 172.21.15.79:0/749299127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc440b4180 0x7fdc440d21f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fdc340097a0 tx=0x7fdc3402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce19ea88d69199a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc48b4e640 1 -- 172.21.15.79:0/749299127 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34037070 con 0x7fdc440b4180 2024-04-26T19:52:18.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc48b4e640 1 -- 172.21.15.79:0/749299127 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc34032440 con 0x7fdc440b4180 2024-04-26T19:52:18.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc48b4e640 1 -- 172.21.15.79:0/749299127 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34032810 con 0x7fdc440b4180 2024-04-26T19:52:18.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/749299127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc440b4180 msgr2=0x7fdc440d21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc4bdb4640 1 --2- 172.21.15.79:0/749299127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc440b4180 0x7fdc440d21f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fdc340097a0 tx=0x7fdc3402a180 comp rx=0 tx=0).stop 2024-04-26T19:52:18.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/749299127 shutdown_connections 2024-04-26T19:52:18.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc4bdb4640 1 --2- 172.21.15.79:0/749299127 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 0x7fdc4415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.558 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc4bdb4640 1 --2- 172.21.15.79:0/749299127 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc440b4180 0x7fdc440d21f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/749299127 >> 172.21.15.79:0/749299127 conn(0x7fdc440b3660 msgr2=0x7fdc44070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:18.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/749299127 shutdown_connections 2024-04-26T19:52:18.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.554+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/749299127 wait complete. 2024-04-26T19:52:18.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.555+0000 7fdc4bdb4640 1 Processor -- start 2024-04-26T19:52:18.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.555+0000 7fdc4bdb4640 1 -- start start 2024-04-26T19:52:18.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4bdb4640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc440b4180 0x7fdc441411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4bdb4640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc44146290 0x7fdc440c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4bdb4640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 0x7fdc4415ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.559 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4bdb4640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44146670 con 0x7fdc44146290 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4bdb4640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7fdc44147180 con 0x7fdc440b4180 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4bdb4640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7fdc440c53e0 con 0x7fdc441469a0 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4934f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc44146290 0x7fdc440c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4934f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc44146290 0x7fdc440c4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40286/0 (socket says 172.21.15.79:40286) 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4934f640 1 -- 172.21.15.79:0/975865818 learned_addr learned my addr 172.21.15.79:0/975865818 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4a351640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 0x7fdc4415ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc49b50640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc440b4180 0x7fdc441411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4934f640 1 -- 172.21.15.79:0/975865818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 msgr2=0x7fdc4415ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4934f640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 0x7fdc4415ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4934f640 1 -- 172.21.15.79:0/975865818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc440b4180 msgr2=0x7fdc441411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.560 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4934f640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc440b4180 0x7fdc441411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4934f640 1 -- 172.21.15.79:0/975865818 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc44140400 con 0x7fdc44146290 2024-04-26T19:52:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.556+0000 7fdc4a351640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 0x7fdc4415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.557+0000 7fdc4934f640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc44146290 0x7fdc440c4d20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdc34035040 tx=0x7fdc340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.557+0000 7fdc3a7fc640 1 -- 172.21.15.79:0/975865818 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34037040 con 0x7fdc44146290 2024-04-26T19:52:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.557+0000 7fdc3a7fc640 1 -- 172.21.15.79:0/975865818 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc34030770 con 0x7fdc44146290 2024-04-26T19:52:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.557+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc441406e0 con 0x7fdc44146290 2024-04-26T19:52:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.557+0000 7fdc3a7fc640 1 -- 172.21.15.79:0/975865818 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34030b40 con 0x7fdc44146290 2024-04-26T19:52:18.561 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.557+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc44140ca0 con 0x7fdc44146290 2024-04-26T19:52:18.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.558+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc440506c0 con 0x7fdc44146290 2024-04-26T19:52:18.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.558+0000 7fdc3a7fc640 1 -- 172.21.15.79:0/975865818 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7fdc34044020 con 0x7fdc44146290 2024-04-26T19:52:18.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.559+0000 7fdc3a7fc640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fdc2806ca10 0x7fdc2806eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.559+0000 7fdc3a7fc640 1 -- 172.21.15.79:0/975865818 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7fdc340ac320 con 0x7fdc44146290 2024-04-26T19:52:18.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.559+0000 7fdc49b50640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fdc2806ca10 0x7fdc2806eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.562 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.559+0000 7fdc49b50640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fdc2806ca10 0x7fdc2806eed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc400012e0 tx=0x7fdc40007ed0 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.564 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.563+0000 7fdc3a7fc640 1 -- 172.21.15.79:0/975865818 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7fdc34032e20 con 0x7fdc44146290 2024-04-26T19:52:18.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.738+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fdc440b1fc0 con 0x7fdc2806ca10 2024-04-26T19:52:18.739 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.739+0000 7fdc3a7fc640 1 -- 172.21.15.79:0/975865818 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdc440b1fc0 con 0x7fdc2806ca10 2024-04-26T19:52:18.739 INFO:teuthology.orchestra.run.smithi079.stderr:instructed osd(s) 7 to deep-scrub 2024-04-26T19:52:18.741 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fdc2806ca10 msgr2=0x7fdc2806eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fdc2806ca10 0x7fdc2806eed0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdc400012e0 tx=0x7fdc40007ed0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc44146290 msgr2=0x7fdc440c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc44146290 0x7fdc440c4d20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdc34035040 tx=0x7fdc340046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 shutdown_connections 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7fdc2806ca10 0x7fdc2806eed0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7fdc441469a0 0x7fdc4415ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7fdc44146290 0x7fdc440c4d20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 --2- 172.21.15.79:0/975865818 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7fdc440b4180 0x7fdc441411c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.740+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 >> 172.21.15.79:0/975865818 conn(0x7fdc440b3660 msgr2=0x7fdc44142f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.741+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 shutdown_connections 2024-04-26T19:52:18.742 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.741+0000 7fdc4bdb4640 1 -- 172.21.15.79:0/975865818 wait complete. 2024-04-26T19:52:18.752 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:52:18.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.847+0000 7f17d795f640 1 Processor -- start 2024-04-26T19:52:18.847 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.847+0000 7f17d795f640 1 -- start start 2024-04-26T19:52:18.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d795f640 1 --2- >> v2:172.21.15.83:3300/0 conn(0x7f17d0156670 0x7f17d015d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d795f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d00cd9c0 0x7f17d00b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d795f640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17d015d670 con 0x7f17d00735a0 2024-04-26T19:52:18.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d795f640 1 -- --> v2:172.21.15.83:3300/0 -- mon_getmap magic: 0 -- 0x7f17d0073110 con 0x7f17d0156670 2024-04-26T19:52:18.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d795f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f17d00ce1a0 con 0x7f17d00cd9c0 2024-04-26T19:52:18.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d5efc640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f17d00735a0 0x7f17d015fca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:33734/0 (socket says 172.21.15.79:33734) 2024-04-26T19:52:18.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d5efc640 1 -- 172.21.15.79:0/3135626459 learned_addr learned my addr 172.21.15.79:0/3135626459 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:18.849 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d56fb640 1 --2- 172.21.15.79:0/3135626459 >> v2:172.21.15.83:3300/0 conn(0x7f17d0156670 0x7f17d015d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d4efa640 1 --2- 172.21.15.79:0/3135626459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d00cd9c0 0x7f17d00b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17c7fff640 1 -- 172.21.15.79:0/3135626459 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1954557289 0 0) 0x7f17c8001490 con 0x7f17d00735a0 2024-04-26T19:52:18.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17c7fff640 1 -- 172.21.15.79:0/3135626459 --> v1:172.21.15.79:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17bc003520 con 0x7f17d00735a0 2024-04-26T19:52:18.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d4efa640 1 -- 172.21.15.79:0/3135626459 >> v2:172.21.15.83:3300/0 conn(0x7f17d0156670 msgr2=0x7f17d015d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:18.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d4efa640 1 --2- 172.21.15.79:0/3135626459 >> v2:172.21.15.83:3300/0 conn(0x7f17d0156670 0x7f17d015d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d4efa640 1 -- 172.21.15.79:0/3135626459 >> v1:172.21.15.79:6789/0 conn(0x7f17d00735a0 legacy=0x7f17d015fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17c7fff640 1 -- 172.21.15.79:0/3135626459 <== mon.0 v1:172.21.15.79:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2002177569 0 0) 0x7f17bc003520 con 0x7f17d00735a0 2024-04-26T19:52:18.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d4efa640 1 -- 172.21.15.79:0/3135626459 --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d00cdec0 con 0x7f17d00cd9c0 2024-04-26T19:52:18.850 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.848+0000 7f17d56fb640 1 --2- 172.21.15.79:0/3135626459 >> v2:172.21.15.83:3300/0 conn(0x7f17d0156670 0x7f17d015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T19:52:18.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.849+0000 7f17d4efa640 1 --2- 172.21.15.79:0/3135626459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d00cd9c0 0x7f17d00b3de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17c00097a0 tx=0x7f17c002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644ed37cd503869f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.849+0000 7f17c7fff640 1 -- 172.21.15.79:0/3135626459 <== mon.2 v2:172.21.15.83:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0037070 con 0x7f17d00cd9c0 2024-04-26T19:52:18.854 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.849+0000 7f17c7fff640 1 -- 172.21.15.79:0/3135626459 <== mon.2 v2:172.21.15.83:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c0032440 con 0x7f17d00cd9c0 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.849+0000 7f17c7fff640 1 -- 172.21.15.79:0/3135626459 <== mon.2 v2:172.21.15.83:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c00327f0 con 0x7f17d00cd9c0 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.850+0000 7f17d795f640 1 -- 172.21.15.79:0/3135626459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d00cd9c0 msgr2=0x7f17d00b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.850+0000 7f17d795f640 1 --2- 172.21.15.79:0/3135626459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d00cd9c0 0x7f17d00b3de0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f17c00097a0 tx=0x7f17c002a180 comp rx=0 tx=0).stop 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.850+0000 7f17d795f640 1 -- 172.21.15.79:0/3135626459 shutdown_connections 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.850+0000 7f17d795f640 1 --2- 172.21.15.79:0/3135626459 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d00cd9c0 0x7f17d00b3de0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.850+0000 7f17d795f640 1 --2- 172.21.15.79:0/3135626459 >> v2:172.21.15.83:3300/0 conn(0x7f17d0156670 0x7f17d015d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.850+0000 7f17d795f640 1 -- 172.21.15.79:0/3135626459 >> 172.21.15.79:0/3135626459 conn(0x7f17d00b39d0 msgr2=0x7f17d006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.850+0000 7f17d795f640 1 -- 172.21.15.79:0/3135626459 shutdown_connections 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.850+0000 7f17d795f640 1 -- 172.21.15.79:0/3135626459 wait complete. 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.851+0000 7f17d795f640 1 Processor -- start 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.851+0000 7f17d795f640 1 -- start start 2024-04-26T19:52:18.855 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d795f640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d00735a0 0x7f17d00b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d795f640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17d00cd9c0 0x7f17d01f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d795f640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d0156670 0x7f17d015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d795f640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0156a50 con 0x7f17d00cd9c0 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d795f640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f17d0072da0 con 0x7f17d00735a0 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d795f640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f17d0156bd0 con 0x7f17d0156670 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d4efa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17d00cd9c0 0x7f17d01f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d4efa640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17d00cd9c0 0x7f17d01f82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:40298/0 (socket says 172.21.15.79:40298) 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d4efa640 1 -- 172.21.15.79:0/3369724364 learned_addr learned my addr 172.21.15.79:0/3369724364 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d56fb640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d00735a0 0x7f17d00b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d5efc640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d0156670 0x7f17d015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d4efa640 1 -- 172.21.15.79:0/3369724364 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d0156670 msgr2=0x7f17d015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.856 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d4efa640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d0156670 0x7f17d015ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d4efa640 1 -- 172.21.15.79:0/3369724364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d00735a0 msgr2=0x7f17d00b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:18.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d4efa640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d00735a0 0x7f17d00b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:18.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d4efa640 1 -- 172.21.15.79:0/3369724364 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d00af830 con 0x7f17d00cd9c0 2024-04-26T19:52:18.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d56fb640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d00735a0 0x7f17d00b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:18.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d4efa640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17d00cd9c0 0x7f17d01f82d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f17c0000c00 tx=0x7f17c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17c5ffb640 1 -- 172.21.15.79:0/3369724364 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0041030 con 0x7f17d00cd9c0 2024-04-26T19:52:18.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17c5ffb640 1 -- 172.21.15.79:0/3369724364 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c003d4e0 con 0x7f17d00cd9c0 2024-04-26T19:52:18.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17c5ffb640 1 -- 172.21.15.79:0/3369724364 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17c0037040 con 0x7f17d00cd9c0 2024-04-26T19:52:18.857 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.852+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d00aea70 con 0x7f17d00cd9c0 2024-04-26T19:52:18.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.853+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d00af030 con 0x7f17d00cd9c0 2024-04-26T19:52:18.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.854+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d00506c0 con 0x7f17d00cd9c0 2024-04-26T19:52:18.858 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.854+0000 7f17c5ffb640 1 -- 172.21.15.79:0/3369724364 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f17c0044070 con 0x7f17d00cd9c0 2024-04-26T19:52:18.859 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.854+0000 7f17c5ffb640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f17b406ca10 0x7f17b406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:18.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.855+0000 7f17c5ffb640 1 -- 172.21.15.79:0/3369724364 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f17c00ac4e0 con 0x7f17d00cd9c0 2024-04-26T19:52:18.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.855+0000 7f17d56fb640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f17b406ca10 0x7f17b406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:18.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.855+0000 7f17d56fb640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f17b406ca10 0x7f17b406eed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009af0 tx=0x7f17cc008040 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:18.860 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:18.859+0000 7f17c5ffb640 1 -- 172.21.15.79:0/3369724364 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f17c0044e40 con 0x7f17d00cd9c0 2024-04-26T19:52:19.029 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.028+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f17d015e550 con 0x7f17b406ca10 2024-04-26T19:52:19.030 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.029+0000 7f17c5ffb640 1 -- 172.21.15.79:0/3369724364 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13355 (secure 0 0 0) 0x7f17d015e550 con 0x7f17b406ca10 2024-04-26T19:52:19.030 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:19.030 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:52:19.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f17b406ca10 msgr2=0x7f17b406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:19.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f17b406ca10 0x7f17b406eed0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17cc009af0 tx=0x7f17cc008040 comp rx=0 tx=0).stop 2024-04-26T19:52:19.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17d00cd9c0 msgr2=0x7f17d01f82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:19.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17d00cd9c0 0x7f17d01f82d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f17c0000c00 tx=0x7f17c00046c0 comp rx=0 tx=0).stop 2024-04-26T19:52:19.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 shutdown_connections 2024-04-26T19:52:19.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f17b406ca10 0x7f17b406eed0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:19.032 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f17d0156670 0x7f17d015ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:19.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f17d00cd9c0 0x7f17d01f82d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:19.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 --2- 172.21.15.79:0/3369724364 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f17d00735a0 0x7f17d00b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:19.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 >> 172.21.15.79:0/3369724364 conn(0x7f17d00b39d0 msgr2=0x7f17d006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:19.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 shutdown_connections 2024-04-26T19:52:19.033 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:19.031+0000 7f17d795f640 1 -- 172.21.15.79:0/3369724364 wait complete. 2024-04-26T19:52:19.042 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-04-26T19:52:17.214651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225548,"kb_used_data":5224,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749172,"statfs":{"total":773094113280,"available":772863152128,"internally_reserved":0,"allocated":5349376,"data_stored":3035552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005370"},"pg_stats":[{"pgid":"1.0","version":"42'139","reported_seq":292,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T19:52:10.859378+0000","last_change":"2024-04-26T19:50:19.373141+0000","last_active":"2024-04-26T19:52:10.859378+0000","last_peered":"2024-04-26T19:52:10.859378+0000","last_clean":"2024-04-26T19:52:10.859378+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:52:10.859378+0000","last_undegraded":"2024-04-26T19:52:10.859378+0000","last_fullsized":"2024-04-26T19:52:10.859378+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T19:50:18.292488+0000","last_clean_scrub_stamp":"2024-04-26T19:50:18.292488+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:19:07.040094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":822220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":822220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:52:19.047 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T19:50:18.292488+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=19, tm_min=50, tm_sec=18, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=19, tm_min=52, tm_sec=14, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T19:52:19.047 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T19:52:30.616 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:52:30.620 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:52:39.048 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T19:52:39.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.156+0000 7f3092b46640 1 Processor -- start 2024-04-26T19:52:39.157 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.156+0000 7f3092b46640 1 -- start start 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f3092b46640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f308c0b4220 0x7f308c1630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f3092b46640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308c144ca0 con 0x7f308c160770 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f3092b46640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f308c163e00 con 0x7f308c160390 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f3092b46640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f308c144f30 con 0x7f308c0b4220 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f308bfff640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f308c0b4220 0x7f308c1630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f30910e3640 1 --1- >> v1:172.21.15.79:6789/0 conn(0x7f308c160770 0x7f308c163610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.79:6789/0 says I am v1:172.21.15.79:43324/0 (socket says 172.21.15.79:43324) 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f30910e3640 1 -- 172.21.15.79:0/418416574 learned_addr learned my addr 172.21.15.79:0/418416574 (peer_addr_for_me v1:172.21.15.79:0/0) 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f308bfff640 1 -- 172.21.15.79:0/418416574 >> v1:172.21.15.83:6789/0 conn(0x7f308c160390 legacy=0x7f308c0b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f308bfff640 1 -- 172.21.15.79:0/418416574 >> v1:172.21.15.79:6789/0 conn(0x7f308c160770 legacy=0x7f308c163610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.158 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f308b7fe640 1 -- 172.21.15.79:0/418416574 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 582419005 0 0) 0x7f308c144ca0 con 0x7f308c160770 2024-04-26T19:52:39.159 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.157+0000 7f308bfff640 1 -- 172.21.15.79:0/418416574 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c145210 con 0x7f308c0b4220 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.158+0000 7f308bfff640 1 --2- 172.21.15.79:0/418416574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 0x7f308c1630d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f30780097a0 tx=0x7f307802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602fa6a1b00c7a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.158+0000 7f308b7fe640 1 -- 172.21.15.79:0/418416574 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078037070 con 0x7f308c0b4220 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.158+0000 7f308b7fe640 1 -- 172.21.15.79:0/418416574 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307802ae50 con 0x7f308c0b4220 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.158+0000 7f308b7fe640 1 -- 172.21.15.79:0/418416574 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3078032660 con 0x7f308c0b4220 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.159+0000 7f3092b46640 1 -- 172.21.15.79:0/418416574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 msgr2=0x7f308c1630d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.159+0000 7f3092b46640 1 --2- 172.21.15.79:0/418416574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 0x7f308c1630d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f30780097a0 tx=0x7f307802a140 comp rx=0 tx=0).stop 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.159+0000 7f3092b46640 1 -- 172.21.15.79:0/418416574 shutdown_connections 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.159+0000 7f3092b46640 1 --2- 172.21.15.79:0/418416574 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 0x7f308c1630d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.159+0000 7f3092b46640 1 -- 172.21.15.79:0/418416574 >> 172.21.15.79:0/418416574 conn(0x7f308c0b2720 msgr2=0x7f308c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.159+0000 7f3092b46640 1 -- 172.21.15.79:0/418416574 shutdown_connections 2024-04-26T19:52:39.160 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.159+0000 7f3092b46640 1 -- 172.21.15.79:0/418416574 wait complete. 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.160+0000 7f3092b46640 1 Processor -- start 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.160+0000 7f3092b46640 1 -- start start 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f3092b46640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 0x7f308c15c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f3092b46640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c160390 0x7f308c1406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f3092b46640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f308c160770 0x7f308c140c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f3092b46640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f308c145b90 con 0x7f308c0b4220 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f3092b46640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f308c145a10 con 0x7f308c160390 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f3092b46640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f308c16a470 con 0x7f308c160770 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30908e2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 0x7f308c15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30908e2640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 0x7f308c15c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46018/0 (socket says 172.21.15.79:46018) 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30908e2640 1 -- 172.21.15.79:0/1354068881 learned_addr learned my addr 172.21.15.79:0/1354068881 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f308bfff640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c160390 0x7f308c1406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.163 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30910e3640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f308c160770 0x7f308c140c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30908e2640 1 -- 172.21.15.79:0/1354068881 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f308c160770 msgr2=0x7f308c140c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30908e2640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f308c160770 0x7f308c140c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30908e2640 1 -- 172.21.15.79:0/1354068881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c160390 msgr2=0x7f308c1406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30908e2640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c160390 0x7f308c1406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30908e2640 1 -- 172.21.15.79:0/1354068881 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308c141460 con 0x7f308c0b4220 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30908e2640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 0x7f308c15c680 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f308000b3e0 tx=0x7f308000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.161+0000 7f30897fa640 1 -- 172.21.15.79:0/1354068881 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080015020 con 0x7f308c0b4220 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.162+0000 7f30897fa640 1 -- 172.21.15.79:0/1354068881 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3080011850 con 0x7f308c0b4220 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.162+0000 7f30897fa640 1 -- 172.21.15.79:0/1354068881 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3080011c40 con 0x7f308c0b4220 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.162+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f308c141740 con 0x7f308c0b4220 2024-04-26T19:52:39.164 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.162+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f308c156dd0 con 0x7f308c0b4220 2024-04-26T19:52:39.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.163+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308c0506c0 con 0x7f308c0b4220 2024-04-26T19:52:39.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.163+0000 7f30897fa640 1 -- 172.21.15.79:0/1354068881 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f3080018020 con 0x7f308c0b4220 2024-04-26T19:52:39.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.163+0000 7f30897fa640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f307006ca10 0x7f307006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.164+0000 7f30897fa640 1 -- 172.21.15.79:0/1354068881 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f308008bd10 con 0x7f308c0b4220 2024-04-26T19:52:39.165 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.164+0000 7f308bfff640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f307006ca10 0x7f307006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.166 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.164+0000 7f308bfff640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f307006ca10 0x7f307006eed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3078034000 tx=0x7f307802e000 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:39.168 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.168+0000 7f30897fa640 1 -- 172.21.15.79:0/1354068881 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f308001f020 con 0x7f308c0b4220 2024-04-26T19:52:39.350 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.350+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 --> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f308c06d7b0 con 0x7f307006ca10 2024-04-26T19:52:39.351 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.350+0000 7f30897fa640 1 -- 172.21.15.79:0/1354068881 <== mgr.5676 v2:172.21.15.79:6832/2762876347 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13358 (secure 0 0 0) 0x7f308c06d7b0 con 0x7f307006ca10 2024-04-26T19:52:39.351 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:39.352 INFO:teuthology.orchestra.run.smithi079.stderr:dumped all 2024-04-26T19:52:39.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.352+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f307006ca10 msgr2=0x7f307006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.353 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.352+0000 7f3092b46640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f307006ca10 0x7f307006eed0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3078034000 tx=0x7f307802e000 comp rx=0 tx=0).stop 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.352+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 msgr2=0x7f308c15c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.352+0000 7f3092b46640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 0x7f308c15c680 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f308000b3e0 tx=0x7f308000b8a0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.353+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 shutdown_connections 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.353+0000 7f3092b46640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f307006ca10 0x7f307006eed0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.353+0000 7f3092b46640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f308c160770 0x7f308c140c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.353+0000 7f3092b46640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f308c160390 0x7f308c1406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.353+0000 7f3092b46640 1 --2- 172.21.15.79:0/1354068881 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f308c0b4220 0x7f308c15c680 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.353+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 >> 172.21.15.79:0/1354068881 conn(0x7f308c0b2720 msgr2=0x7f308c143c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.353+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 shutdown_connections 2024-04-26T19:52:39.354 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.353+0000 7f3092b46640 1 -- 172.21.15.79:0/1354068881 wait complete. 2024-04-26T19:52:39.364 INFO:teuthology.orchestra.run.smithi079.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-04-26T19:52:39.219097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":225540,"kb_used_data":5224,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749180,"statfs":{"total":773094113280,"available":772863160320,"internally_reserved":0,"allocated":5349376,"data_stored":3035552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002515"},"pg_stats":[{"pgid":"1.0","version":"44'140","reported_seq":303,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T19:52:19.122390+0000","last_change":"2024-04-26T19:52:19.122390+0000","last_active":"2024-04-26T19:52:19.122390+0000","last_peered":"2024-04-26T19:52:19.122390+0000","last_clean":"2024-04-26T19:52:19.122390+0000","last_became_active":"2024-04-26T19:50:19.372180+0000","last_became_peered":"2024-04-26T19:50:19.372180+0000","last_unstale":"2024-04-26T19:52:19.122390+0000","last_undegraded":"2024-04-26T19:52:19.122390+0000","last_fullsized":"2024-04-26T19:52:19.122390+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'140","last_scrub_stamp":"2024-04-26T19:52:19.122183+0000","last_deep_scrub":"44'140","last_deep_scrub_stamp":"2024-04-26T19:52:19.122183+0000","last_clean_scrub_stamp":"2024-04-26T19:52:19.122183+0000","objects_scrubbed":2,"log_size":140,"log_dups_size":0,"ondisk_log_size":140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:23:27.872631+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1114112,"data_stored":822220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27528,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31688,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27532,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":520192,"data_stored":231852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28104,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343736,"statfs":{"total":96636764160,"available":96607985664,"internally_reserved":0,"allocated":1114112,"data_stored":822220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T19:52:39.365 DEBUG:teuthology.orchestra.run.smithi079:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T19:52:39.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.464+0000 7f5204dda640 1 Processor -- start 2024-04-26T19:52:39.464 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.464+0000 7f5204dda640 1 -- start start 2024-04-26T19:52:39.466 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f5204dda640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f52000aeeb0 0x7f520006f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f5204dda640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000af290 0x7f52000af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f5204dda640 1 -- --> v1:172.21.15.79:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f520006fa10 con 0x7f5200070160 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f5204dda640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f5200072ea0 con 0x7f52000aeeb0 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f5204dda640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f5200073180 con 0x7f52000af290 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f51fe59c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f52000aeeb0 0x7f520006f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f51fe59c640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f52000aeeb0 0x7f520006f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46028/0 (socket says 172.21.15.79:46028) 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f51fe59c640 1 -- 172.21.15.79:0/1031094708 learned_addr learned my addr 172.21.15.79:0/1031094708 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f51fed9d640 1 --2- 172.21.15.79:0/1031094708 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000af290 0x7f52000af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f51fe59c640 1 -- 172.21.15.79:0/1031094708 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000af290 msgr2=0x7f52000af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f51fe59c640 1 --2- 172.21.15.79:0/1031094708 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000af290 0x7f52000af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f51fe59c640 1 -- 172.21.15.79:0/1031094708 >> v1:172.21.15.79:6789/0 conn(0x7f5200070160 legacy=0x7f52000b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f51fd59a640 1 -- 172.21.15.79:0/1031094708 <== mon.0 v1:172.21.15.79:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 657203786 0 0) 0x7f520006fa10 con 0x7f5200070160 2024-04-26T19:52:39.467 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.465+0000 7f51fe59c640 1 -- 172.21.15.79:0/1031094708 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5200073460 con 0x7f52000aeeb0 2024-04-26T19:52:39.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.466+0000 7f51fe59c640 1 --2- 172.21.15.79:0/1031094708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000aeeb0 0x7f520006f0e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009870 tx=0x7f51f002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28cab4618511892 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:39.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.466+0000 7f51fd59a640 1 -- 172.21.15.79:0/1031094708 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0037070 con 0x7f52000aeeb0 2024-04-26T19:52:39.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.466+0000 7f51fd59a640 1 -- 172.21.15.79:0/1031094708 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f0032440 con 0x7f52000aeeb0 2024-04-26T19:52:39.468 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.466+0000 7f51fd59a640 1 -- 172.21.15.79:0/1031094708 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f0032810 con 0x7f52000aeeb0 2024-04-26T19:52:39.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.467+0000 7f5204dda640 1 -- 172.21.15.79:0/1031094708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000aeeb0 msgr2=0x7f520006f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.467+0000 7f5204dda640 1 --2- 172.21.15.79:0/1031094708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000aeeb0 0x7f520006f0e0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009870 tx=0x7f51f002a1c0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.467+0000 7f5204dda640 1 -- 172.21.15.79:0/1031094708 shutdown_connections 2024-04-26T19:52:39.469 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.467+0000 7f5204dda640 1 --2- 172.21.15.79:0/1031094708 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000af290 0x7f52000af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.467+0000 7f5204dda640 1 --2- 172.21.15.79:0/1031094708 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000aeeb0 0x7f520006f0e0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.467+0000 7f5204dda640 1 -- 172.21.15.79:0/1031094708 >> 172.21.15.79:0/1031094708 conn(0x7f52000687e0 msgr2=0x7f5200071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.467+0000 7f5204dda640 1 -- 172.21.15.79:0/1031094708 shutdown_connections 2024-04-26T19:52:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.467+0000 7f5204dda640 1 -- 172.21.15.79:0/1031094708 wait complete. 2024-04-26T19:52:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.468+0000 7f5204dda640 1 Processor -- start 2024-04-26T19:52:39.470 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.468+0000 7f5204dda640 1 -- start start 2024-04-26T19:52:39.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f5204dda640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200070160 0x7f5200159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f5204dda640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000aeeb0 0x7f5200132600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f5204dda640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000af290 0x7f5200132b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f5204dda640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f52000b1b60 con 0x7f52000af290 2024-04-26T19:52:39.473 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f5204dda640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f5200149380 con 0x7f5200070160 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f5204dda640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f52000afc00 con 0x7f52000aeeb0 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000af290 0x7f5200132b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fed9d640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000af290 0x7f5200132b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46044/0 (socket says 172.21.15.79:46044) 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fed9d640 1 -- 172.21.15.79:0/1016745886 learned_addr learned my addr 172.21.15.79:0/1016745886 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fdd9b640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000aeeb0 0x7f5200132600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fe59c640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200070160 0x7f5200159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fed9d640 1 -- 172.21.15.79:0/1016745886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000aeeb0 msgr2=0x7f5200132600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fed9d640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000aeeb0 0x7f5200132600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fed9d640 1 -- 172.21.15.79:0/1016745886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200070160 msgr2=0x7f5200159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fed9d640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200070160 0x7f5200159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fed9d640 1 -- 172.21.15.79:0/1016745886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f520015a530 con 0x7f52000af290 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.469+0000 7f51fdd9b640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000aeeb0 0x7f5200132600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:39.474 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.470+0000 7f51fed9d640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000af290 0x7f5200132b40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f51f8007e40 tx=0x7f51f800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.470+0000 7f51eeffd640 1 -- 172.21.15.79:0/1016745886 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8014620 con 0x7f52000af290 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.470+0000 7f51eeffd640 1 -- 172.21.15.79:0/1016745886 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51f8019070 con 0x7f52000af290 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.470+0000 7f51eeffd640 1 -- 172.21.15.79:0/1016745886 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51f8014a20 con 0x7f52000af290 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.470+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52001307c0 con 0x7f52000af290 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.470+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5200130d80 con 0x7f52000af290 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.471+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52000506c0 con 0x7f52000af290 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.471+0000 7f51eeffd640 1 -- 172.21.15.79:0/1016745886 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f51f8014bc0 con 0x7f52000af290 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.472+0000 7f51eeffd640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f51e006ca10 0x7f51e006eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.472+0000 7f51fe59c640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f51e006ca10 0x7f51e006eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.472+0000 7f51eeffd640 1 -- 172.21.15.79:0/1016745886 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f51f808c620 con 0x7f52000af290 2024-04-26T19:52:39.475 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.472+0000 7f51fe59c640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f51e006ca10 0x7f51e006eed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51f0034000 tx=0x7f51f002e000 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:39.480 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.480+0000 7f51eeffd640 1 -- 172.21.15.79:0/1016745886 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f520015a530 con 0x7f52000af290 2024-04-26T19:52:39.666 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.665+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f520006a800 con 0x7f52000af290 2024-04-26T19:52:39.670 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.670+0000 7f51eeffd640 1 -- 172.21.15.79:0/1016745886 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) ==== 123+0+0 (secure 0 0 0) 0x7f51f805a8e0 con 0x7f52000af290 2024-04-26T19:52:39.672 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.672+0000 7f51eeffd640 1 -- 172.21.15.79:0/1016745886 <== mon.0 v2:172.21.15.79:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f51f805a1e0 con 0x7f52000af290 2024-04-26T19:52:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f51e006ca10 msgr2=0x7f51e006eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f51e006ca10 0x7f51e006eed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f51f0034000 tx=0x7f51f002e000 comp rx=0 tx=0).stop 2024-04-26T19:52:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000af290 msgr2=0x7f5200132b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000af290 0x7f5200132b40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f51f8007e40 tx=0x7f51f800e4b0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 shutdown_connections 2024-04-26T19:52:39.674 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f51e006ca10 0x7f51e006eed0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f52000af290 0x7f5200132b40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f52000aeeb0 0x7f5200132600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 --2- 172.21.15.79:0/1016745886 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f5200070160 0x7f5200159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 >> 172.21.15.79:0/1016745886 conn(0x7f52000687e0 msgr2=0x7f52000b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 shutdown_connections 2024-04-26T19:52:39.675 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.673+0000 7f5204dda640 1 -- 172.21.15.79:0/1016745886 wait complete. 2024-04-26T19:52:39.685 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T19:52:39.686 DEBUG:teuthology.orchestra.run.smithi079:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T19:52:39.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.783+0000 7f4a3dd0a640 1 Processor -- start 2024-04-26T19:52:39.784 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.783+0000 7f4a3dd0a640 1 -- start start 2024-04-26T19:52:39.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a3dd0a640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4a3815d440 0x7f4a3815d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.785 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a3dd0a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 0x7f4a38072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a3dd0a640 1 -- --> v1:172.21.15.83:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a380734f0 con 0x7f4a380b3de0 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a3dd0a640 1 -- --> v2:172.21.15.79:3300/0 -- mon_getmap magic: 0 -- 0x7f4a380737d0 con 0x7f4a3815d440 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a3dd0a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4a380cde30 con 0x7f4a38072a30 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a377fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4a3815d440 0x7f4a3815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a377fe640 1 --2- >> v2:172.21.15.79:3300/0 conn(0x7f4a3815d440 0x7f4a3815d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46058/0 (socket says 172.21.15.79:46058) 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a377fe640 1 -- 172.21.15.79:0/657310944 learned_addr learned my addr 172.21.15.79:0/657310944 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a36ffd640 1 --2- 172.21.15.79:0/657310944 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 0x7f4a38072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a377fe640 1 -- 172.21.15.79:0/657310944 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 msgr2=0x7f4a38072e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a377fe640 1 --2- 172.21.15.79:0/657310944 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 0x7f4a38072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a377fe640 1 -- 172.21.15.79:0/657310944 >> v1:172.21.15.83:6789/0 conn(0x7f4a380b3de0 legacy=0x7f4a380ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.784+0000 7f4a377fe640 1 -- 172.21.15.79:0/657310944 --> v2:172.21.15.79:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a380b3230 con 0x7f4a3815d440 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.785+0000 7f4a377fe640 1 --2- 172.21.15.79:0/657310944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a3815d440 0x7f4a3815d820 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009870 tx=0x7f4a2802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de98f18a9bd36800 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:39.786 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.785+0000 7f4a367fc640 1 -- 172.21.15.79:0/657310944 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28037070 con 0x7f4a3815d440 2024-04-26T19:52:39.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.785+0000 7f4a367fc640 1 -- 172.21.15.79:0/657310944 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4a28032440 con 0x7f4a3815d440 2024-04-26T19:52:39.787 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.785+0000 7f4a367fc640 1 -- 172.21.15.79:0/657310944 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28032830 con 0x7f4a3815d440 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.786+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/657310944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a3815d440 msgr2=0x7f4a3815d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.786+0000 7f4a3dd0a640 1 --2- 172.21.15.79:0/657310944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a3815d440 0x7f4a3815d820 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009870 tx=0x7f4a2802a210 comp rx=0 tx=0).stop 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.786+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/657310944 shutdown_connections 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.786+0000 7f4a3dd0a640 1 --2- 172.21.15.79:0/657310944 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 0x7f4a38072e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.786+0000 7f4a3dd0a640 1 --2- 172.21.15.79:0/657310944 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a3815d440 0x7f4a3815d820 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.786+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/657310944 >> 172.21.15.79:0/657310944 conn(0x7f4a380b39d0 msgr2=0x7f4a3806feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.786+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/657310944 shutdown_connections 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.786+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/657310944 wait complete. 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.787+0000 7f4a3dd0a640 1 Processor -- start 2024-04-26T19:52:39.788 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.787+0000 7f4a3dd0a640 1 -- start start 2024-04-26T19:52:39.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a3dd0a640 1 --2- >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 0x7f4a38149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.790 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a3dd0a640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a380b3de0 0x7f4a381f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a3dd0a640 1 --2- >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3815d440 0x7f4a38158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a3dd0a640 1 -- --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_getmap magic: 0 -- 0x7f4a380b4270 con 0x7f4a380b3de0 2024-04-26T19:52:39.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a3dd0a640 1 -- --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3815dd60 con 0x7f4a3815d440 2024-04-26T19:52:39.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a3dd0a640 1 -- --> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] -- mon_getmap magic: 0 -- 0x7f4a380b43f0 con 0x7f4a38072a30 2024-04-26T19:52:39.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a36ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a380b3de0 0x7f4a381f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.791 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a36ffd640 1 --2- >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a380b3de0 0x7f4a381f4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.79:3300/0 says I am v2:172.21.15.79:46072/0 (socket says 172.21.15.79:46072) 2024-04-26T19:52:39.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a36ffd640 1 -- 172.21.15.79:0/2913331040 learned_addr learned my addr 172.21.15.79:0/2913331040 (peer_addr_for_me v2:172.21.15.79:0/0) 2024-04-26T19:52:39.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a377fe640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 0x7f4a38149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.793 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a37fff640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3815d440 0x7f4a38158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a36ffd640 1 -- 172.21.15.79:0/2913331040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 msgr2=0x7f4a38149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a36ffd640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 0x7f4a38149fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a36ffd640 1 -- 172.21.15.79:0/2913331040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3815d440 msgr2=0x7f4a38158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a36ffd640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3815d440 0x7f4a38158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a36ffd640 1 -- 172.21.15.79:0/2913331040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a380b0550 con 0x7f4a380b3de0 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a37fff640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3815d440 0x7f4a38158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a36ffd640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a380b3de0 0x7f4a381f4fd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00c7f0 tx=0x7f4a2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a1ffff640 1 -- 172.21.15.79:0/2913331040 <== mon.0 v2:172.21.15.79:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c015070 con 0x7f4a380b3de0 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a1ffff640 1 -- 172.21.15.79:0/2913331040 <== mon.0 v2:172.21.15.79:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f4a2c004240 con 0x7f4a380b3de0 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a38151320 con 0x7f4a380b3de0 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a1ffff640 1 -- 172.21.15.79:0/2913331040 <== mon.0 v2:172.21.15.79:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c007620 con 0x7f4a380b3de0 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.788+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a380b0ad0 con 0x7f4a380b3de0 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.789+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a380506c0 con 0x7f4a380b3de0 2024-04-26T19:52:39.794 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.790+0000 7f4a1ffff640 1 -- 172.21.15.79:0/2913331040 <== mon.0 v2:172.21.15.79:3300/0 4 ==== mgrmap(e 15) ==== 91142+0+0 (secure 0 0 0) 0x7f4a2c0077e0 con 0x7f4a380b3de0 2024-04-26T19:52:39.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.790+0000 7f4a1ffff640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f4a2406ca10 0x7f4a2406eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T19:52:39.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.791+0000 7f4a1ffff640 1 -- 172.21.15.79:0/2913331040 <== mon.0 v2:172.21.15.79:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5937+0+0 (secure 0 0 0) 0x7f4a2c08c0b0 con 0x7f4a380b3de0 2024-04-26T19:52:39.795 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.791+0000 7f4a377fe640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f4a2406ca10 0x7f4a2406eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T19:52:39.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.791+0000 7f4a377fe640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f4a2406ca10 0x7f4a2406eed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4a28034000 tx=0x7f4a2802e000 comp rx=0 tx=0).ready entity=mgr.5676 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T19:52:39.796 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:39.795+0000 7f4a1ffff640 1 -- 172.21.15.79:0/2913331040 <== mon.0 v2:172.21.15.79:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187370 (secure 0 0 0) 0x7f4a2c059b80 con 0x7f4a380b3de0 2024-04-26T19:52:40.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.015+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 --> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a380cfe10 con 0x7f4a380b3de0 2024-04-26T19:52:40.016 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.016+0000 7f4a1ffff640 1 -- 172.21.15.79:0/2913331040 <== mon.0 v2:172.21.15.79:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3866 (secure 0 0 0) 0x7f4a2c05de80 con 0x7f4a380b3de0 2024-04-26T19:52:40.017 INFO:teuthology.orchestra.run.smithi079.stdout: 2024-04-26T19:52:40.017 INFO:teuthology.orchestra.run.smithi079.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4802,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6835/2061494256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6833","nonce":2061494256},{"type":"v1","addr":"172.21.15.83:6835","nonce":2061494256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5110,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.83:6834/4034640420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.83:6832","nonce":4034640420},{"type":"v1","addr":"172.21.15.83:6834","nonce":4034640420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5340,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6836/238749789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6834","nonce":238749789},{"type":"v1","addr":"172.21.15.79:6836","nonce":238749789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.79:6837/1301973444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.79:6835","nonce":1301973444},{"type":"v1","addr":"172.21.15.79:6837","nonce":1301973444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18}],"filesystems":[]} 2024-04-26T19:52:40.017 INFO:teuthology.orchestra.run.smithi079.stderr:dumped fsmap epoch 18 2024-04-26T19:52:40.018 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.017+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f4a2406ca10 msgr2=0x7f4a2406eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.017+0000 7f4a3dd0a640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f4a2406ca10 0x7f4a2406eed0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4a28034000 tx=0x7f4a2802e000 comp rx=0 tx=0).stop 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.017+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a380b3de0 msgr2=0x7f4a381f4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.017+0000 7f4a3dd0a640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a380b3de0 0x7f4a381f4fd0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c00c7f0 tx=0x7f4a2c00ccb0 comp rx=0 tx=0).stop 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.018+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 shutdown_connections 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.018+0000 7f4a3dd0a640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:6832/2762876347,v1:172.21.15.79:6833/2762876347] conn(0x7f4a2406ca10 0x7f4a2406eed0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.018+0000 7f4a3dd0a640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] conn(0x7f4a3815d440 0x7f4a38158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.018+0000 7f4a3dd0a640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.79:3300/0,v1:172.21.15.79:6789/0] conn(0x7f4a380b3de0 0x7f4a381f4fd0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.018+0000 7f4a3dd0a640 1 --2- 172.21.15.79:0/2913331040 >> [v2:172.21.15.83:3301/0,v1:172.21.15.83:6790/0] conn(0x7f4a38072a30 0x7f4a38149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.018+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 >> 172.21.15.79:0/2913331040 conn(0x7f4a380b39d0 msgr2=0x7f4a3806f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.018+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 shutdown_connections 2024-04-26T19:52:40.019 INFO:teuthology.orchestra.run.smithi079.stderr:2024-04-26T19:52:40.018+0000 7f4a3dd0a640 1 -- 172.21.15.79:0/2913331040 wait complete. 2024-04-26T19:52:40.029 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-26T19:52:40.029 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T19:52:40.030 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-26T19:52:40.030 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.039 INFO:tasks.ceph.mds.a:Stopped 2024-04-26T19:52:40.039 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-26T19:52:40.039 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.089 INFO:tasks.ceph.mds.c:Stopped 2024-04-26T19:52:40.089 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-26T19:52:40.089 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.101 INFO:tasks.ceph.mds.b:Stopped 2024-04-26T19:52:40.101 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-26T19:52:40.102 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.154 INFO:tasks.ceph.mds.d:Stopped 2024-04-26T19:52:40.154 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T19:52:40.155 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T19:52:40.155 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.186 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T19:52:40.186 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T19:52:40.186 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.266 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T19:52:40.266 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T19:52:40.266 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.341 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T19:52:40.341 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-26T19:52:40.341 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.411 INFO:tasks.ceph.osd.3:Stopped 2024-04-26T19:52:40.411 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-26T19:52:40.412 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.447 INFO:tasks.ceph.osd.4:Stopped 2024-04-26T19:52:40.447 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-26T19:52:40.447 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.515 INFO:tasks.ceph.osd.5:Stopped 2024-04-26T19:52:40.515 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-26T19:52:40.515 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.585 INFO:tasks.ceph.osd.6:Stopped 2024-04-26T19:52:40.586 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-26T19:52:40.586 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.653 INFO:tasks.ceph.osd.7:Stopped 2024-04-26T19:52:40.653 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T19:52:40.653 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-26T19:52:40.653 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.681 INFO:tasks.ceph.mgr.y:Stopped 2024-04-26T19:52:40.681 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T19:52:40.681 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.720 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T19:52:40.720 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T19:52:40.720 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T19:52:40.720 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.732 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T19:52:40.732 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T19:52:40.732 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.773 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T19:52:40.773 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T19:52:40.773 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T19:52:40.825 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T19:52:40.825 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T19:52:40.826 DEBUG:teuthology.orchestra.run.smithi079:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T19:52:40.856 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:52:40.856 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T19:52:41.084 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:52:41.084 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T19:52:41.181 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:52:41.181 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T19:52:41.235 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:52:41.236 DEBUG:teuthology.orchestra.run.smithi079:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-26T19:52:41.288 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:52:41.289 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-26T19:52:41.473 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:52:41.473 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-26T19:52:41.526 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:52:41.527 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-26T19:52:41.578 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:52:41.579 DEBUG:teuthology.orchestra.run.smithi083:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-26T19:52:41.636 INFO:tasks.ceph:Archiving mon data... 2024-04-26T19:52:41.637 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/data/mon.a.tgz 2024-04-26T19:52:41.637 DEBUG:teuthology.orchestra.run.smithi079:> mktemp 2024-04-26T19:52:41.655 INFO:teuthology.orchestra.run.smithi079.stdout:/tmp/tmp.v6js89Bb9N 2024-04-26T19:52:41.656 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.v6js89Bb9N 2024-04-26T19:52:41.843 DEBUG:teuthology.orchestra.run.smithi079:> sudo chmod 0666 /tmp/tmp.v6js89Bb9N 2024-04-26T19:52:41.925 DEBUG:teuthology.orchestra.remote:smithi079:/tmp/tmp.v6js89Bb9N is 723KB 2024-04-26T19:52:42.030 DEBUG:teuthology.orchestra.run.smithi079:> rm -fr /tmp/tmp.v6js89Bb9N 2024-04-26T19:52:42.046 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/data/mon.b.tgz 2024-04-26T19:52:42.047 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-04-26T19:52:42.066 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.zRFfu7o5SU 2024-04-26T19:52:42.066 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.zRFfu7o5SU 2024-04-26T19:52:42.332 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.zRFfu7o5SU 2024-04-26T19:52:42.414 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.zRFfu7o5SU is 1MB 2024-04-26T19:52:42.527 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.zRFfu7o5SU 2024-04-26T19:52:42.542 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/data/mon.c.tgz 2024-04-26T19:52:42.543 DEBUG:teuthology.orchestra.run.smithi083:> mktemp 2024-04-26T19:52:42.595 INFO:teuthology.orchestra.run.smithi083.stdout:/tmp/tmp.xGyQXVW8w6 2024-04-26T19:52:42.596 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.xGyQXVW8w6 2024-04-26T19:52:42.822 DEBUG:teuthology.orchestra.run.smithi083:> sudo chmod 0666 /tmp/tmp.xGyQXVW8w6 2024-04-26T19:52:42.902 DEBUG:teuthology.orchestra.remote:smithi083:/tmp/tmp.xGyQXVW8w6 is 1MB 2024-04-26T19:52:43.010 DEBUG:teuthology.orchestra.run.smithi083:> rm -fr /tmp/tmp.xGyQXVW8w6 2024-04-26T19:52:43.025 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T19:52:43.025 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T19:52:43.028 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T19:52:43.112 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T19:52:43.145 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T19:52:43.145 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T19:52:43.146 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/remote/smithi079/crash 2024-04-26T19:52:43.147 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T19:52:43.179 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/remote/smithi083/crash 2024-04-26T19:52:43.180 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T19:52:43.207 INFO:tasks.ceph:Shutting down logrotate 2024-04-26T19:52:43.207 DEBUG:teuthology.orchestra.run.smithi079:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:52:43.225 DEBUG:teuthology.orchestra.run.smithi083:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-26T19:52:43.299 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T19:52:43.325 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-26T19:52:43.348 INFO:tasks.ceph:Compressing logs... 2024-04-26T19:52:43.348 DEBUG:teuthology.orchestra.run.smithi079:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T19:52:43.368 DEBUG:teuthology.orchestra.run.smithi083:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T19:52:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34470.log 2024-04-26T19:52:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34550.log 2024-04-26T19:52:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T19:52:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34470.log: /var/log/ceph/ceph-client.admin.34550.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T19:52:43.435 INFO:teuthology.orchestra.run.smithi083.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34470.log.gz 2024-04-26T19:52:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34550.log.gz 2024-04-26T19:52:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-26T19:52:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-26T19:52:43.435 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41366.log 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mds.b.log: 91.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34681.log 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.3.41366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41366.log.gz 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34681.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34601.log 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.4.log: 87.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.34681.log.gz --verbose 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr: -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34601.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mgr.x.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34601.log.gz 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40966.log 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-26T19:52:43.436 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.1.40966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40961.log 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40966.log.gz 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41362.log 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.1.40961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34641.log 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.40961.log.gz 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.3.41362.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41362.log.gz 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34510.log 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34761.log 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34641.log.gz 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34510.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34510.log.gz 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.34761.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34761.log.gz 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34721.log 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-mon.b.log: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T19:52:43.437 INFO:teuthology.orchestra.run.smithi083.stderr:/var/log/ceph/ceph-client.admin.34721.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34721.log.gz 2024-04-26T19:52:43.459 INFO:teuthology.orchestra.run.smithi083.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-26T19:52:43.519 INFO:teuthology.orchestra.run.smithi083.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T19:52:43.520 INFO:teuthology.orchestra.run.smithi083.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T19:52:43.529 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41502.log 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44208.log 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:gzip/var/log/ceph/ceph-client.admin.41502.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.48489.log 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46410.log 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41502.log.gz 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48489.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44208.log.gz 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48927.log 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48489.log.gz 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49677.log 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52403.log 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46410.log.gz 2024-04-26T19:52:43.530 INFO:teuthology.orchestra.run.smithi079.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48927.log.gz 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48657.log 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49677.log.gz 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48657.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52403.log.gz 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50156.log 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48657.log.gz 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52643.log 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49635.log 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52643.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50156.log.gz 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45626.log 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49635.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52643.log.gz 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53757.log 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49635.log.gz 2024-04-26T19:52:43.531 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.0.45626.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.0.45626.log.gz -- 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr: /var/log/ceph/ceph-client.0.50830.log 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53463.log 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.0.50830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46242.log 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53757.log.gz 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50830.log.gz 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43534.log 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53463.log.gz 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47529.log 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46242.log.gz 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42816.log 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53589.log 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43534.log.gz 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42816.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47529.log.gz 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42422.log 2024-04-26T19:52:43.532 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53589.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42816.log.gz 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42732.log 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52003.log 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53589.log.gz 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42732.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42422.log.gz 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51783.log 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43871.log 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42732.log.gz 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52003.log.gz 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53085.log 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46452.log 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51783.log.gz 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43871.log.gz 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52517.log 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46452.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53085.log.gz 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43859.log 2024-04-26T19:52:43.533 INFO:teuthology.orchestra.run.smithi079.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46452.log.gz 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52517.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43859.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52517.log.gz 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42548.log 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45302.log 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43859.log.gz 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49508.log 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42548.log.gz 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.45302.log: gzip -5 --verbose 84.4% -- -- replaced with /var/log/ceph/ceph.log.gz /var/log/ceph/ceph-client.admin.43874.log 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49508.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45302.log.gz 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53673.log 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43874.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49508.log.gz 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44616.log 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53673.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43874.log.gz 2024-04-26T19:52:43.534 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43873.log 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53673.log.gz 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43535.log 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41374.log 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44616.log.gz 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43535.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43873.log.gz 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41374.log: /var/log/ceph/ceph-client.admin.52559.log 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43535.log.gz 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49466.log 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41374.log.gz 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51004.log 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49466.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52559.log.gz 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49424.log 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49466.log.gz 2024-04-26T19:52:43.535 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50534.log 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49424.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51004.log.gz 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52727.log 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50534.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49424.log.gz 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47250.log 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50534.log.gz 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47124.log 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52727.log: /var/log/ceph/ceph-client.admin.47250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44198.log 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52727.log.gz 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48405.log 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47250.log.gz 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47124.log.gz 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52119.log 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48405.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44198.log.gz 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50026.log 2024-04-26T19:52:43.536 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52119.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48405.log.gz 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52119.log.gz 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43527.log 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.audit.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50026.log.gz 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41228.log 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49845.log 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41228.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43527.log.gz 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51877.log 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41228.log.gz 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43870.log 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49845.log.gz 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52917.log 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51877.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53421.log 2024-04-26T19:52:43.537 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51877.log.gz 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52917.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43870.log.gz 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43230.log 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53421.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52917.log.gz 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50240.log 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43230.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53421.log.gz 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51062.log 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50240.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43230.log.gz 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52319.log 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51062.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50240.log.gz 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42380.log 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52319.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51062.log.gz 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52319.log.gz 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47166.log 2024-04-26T19:52:43.538 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42380.log.gz 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41670.log 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz/var/log/ceph/ceph-client.admin.47166.log: 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46326.log 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47166.log.gz 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49889.log 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46326.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41670.log.gz 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47571.log 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46326.log.gz 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44658.log 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47571.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49889.log.gz 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43188.log 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44658.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47571.log.gz 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34623.log 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44658.log.gz 2024-04-26T19:52:43.539 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42648.log 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34623.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43188.log.gz 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40662.log 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42648.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34623.log.gz 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43536.log 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42648.log.gz 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.40662.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46872.log 88.5% 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40662.log.gz 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.3.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43536.log.gz 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52277.log 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42690.log 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46620.log 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46872.log.gz 2024-04-26T19:52:43.540 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52277.log.gz 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53295.log 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46620.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42690.log.gz 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46620.log.gz 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46063.log 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53295.log.gz 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51380.log 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50408.log 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.2.46063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46063.log.gz 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47655.log 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51380.log.gz 2024-04-26T19:52:43.541 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41016.log 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47655.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50408.log.gz 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48159.log 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47655.log.gz 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47859.log 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41016.log: /var/log/ceph/ceph-client.admin.48159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44844.log 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48159.log.gz 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41016.log.gz 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41058.log 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0%/var/log/ceph/ceph-client.admin.44844.log: -- replaced with /var/log/ceph/ceph-client.admin.47859.log.gz 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41058.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44844.log.gz 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43865.log 2024-04-26T19:52:43.542 INFO:teuthology.orchestra.run.smithi079.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41058.log.gz 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.43865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50450.log 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43865.log.gz 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45621.log 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46494.log 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50450.log.gz 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.0.45621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41544.log 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45621.log.gz 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41778.log 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46494.log.gz 2024-04-26T19:52:43.543 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52361.log 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5/var/log/ceph/ceph-client.admin.41778.log: 88.8% --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41544.log.gz 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52361.log: gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41778.log.gz 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46914.log 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-osd.0.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52361.log.gz 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45012.log 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50912.log 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.45012.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46914.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43062.log 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45012.log.gz 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51485.log 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50912.log: /var/log/ceph/ceph-client.admin.43062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45138.log 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51485.log: 89.1% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50912.log.gz 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43062.log.gz 2024-04-26T19:52:43.544 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51485.log.gz 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46998.log 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45138.log: -- /var/log/ceph/ceph-client.admin.46788.log 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43529.log 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46998.log.gz 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45138.log.gz 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46788.log: gzip -5 --verbose -- 88.1% /var/log/ceph/ceph-client.admin.42590.log 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46788.log.gz 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53908.log 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43529.log.gz 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49593.log 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53715.log 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42590.log.gz 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53908.log: /var/log/ceph/ceph-client.admin.49593.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53908.log.gz 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49593.log.gz 2024-04-26T19:52:43.545 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43869.log 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48531.log 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9%/var/log/ceph/ceph-client.admin.43869.log: -- replaced with /var/log/ceph/ceph-client.admin.53715.log.gz 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52601.log 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43869.log.gz 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49221.log 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48531.log: /var/log/ceph/ceph-client.admin.52601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49803.log 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49221.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52601.log.gz 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48531.log.gz 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49221.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41184.log 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46704.log 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41184.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49803.log.gz 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49984.log 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46704.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41184.log.gz 2024-04-26T19:52:43.546 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43272.log 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46704.log.gz 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44802.log 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49984.log.gz 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43020.log 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0%/var/log/ceph/ceph-client.admin.44802.log: -- replaced with /var/log/ceph/ceph-client.admin.43272.log.gz 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41460.log 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44802.log.gz 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49933.log 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43020.log: /var/log/ceph/ceph-client.admin.41460.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49933.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43020.log.gz 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41460.log.gz 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49933.log.gz 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48615.log 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48699.log 2024-04-26T19:52:43.547 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48615.log.gz 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53211.log 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42193.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48699.log.gz 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47376.log 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53211.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53211.log.gz 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose 92.8% -- /var/log/ceph/ceph-client.admin.44406.log 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47376.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47376.log.gz 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43537.log 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49095.log 2024-04-26T19:52:43.548 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43537.log: /var/log/ceph/ceph-client.admin.44406.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44406.log.gz 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43537.log.gz 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41272.log 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40973.log 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49095.log: /var/log/ceph/ceph-client.admin.41272.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49095.log.gz 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41272.log.gz 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49719.log 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.40973.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40973.log.gz 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42774.log 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44490.log 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49719.log: /var/log/ceph/ceph-client.admin.42774.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49719.log.gz 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42774.log.gz 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52045.log 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44490.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44490.log.gz 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47613.log 2024-04-26T19:52:43.549 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43532.log 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52045.log: /var/log/ceph/ceph-client.admin.47613.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52045.log.gz 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47613.log.gz 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53858.log 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43532.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43532.log.gz 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50282.log 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44574.log 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53858.log: /var/log/ceph/ceph-client.admin.50282.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53858.log.gz 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50282.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47208.log 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44574.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44574.log.gz 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52463.log 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44532.log 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52463.log: /var/log/ceph/ceph-client.admin.47208.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52463.log.gz 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47208.log.gz 2024-04-26T19:52:43.550 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41586.log 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53169.log 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44532.log: /var/log/ceph/ceph-client.admin.41586.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44532.log.gz 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41586.log.gz 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48117.log 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53169.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53169.log.gz 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50834.log 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53127.log 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48117.log: /var/log/ceph/ceph-client.0.50834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50834.log.gz 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48117.log.gz 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47292.log 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48969.log 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53127.log: /var/log/ceph/ceph-client.admin.47292.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53127.log.gz 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47292.log.gz 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44718.log 2024-04-26T19:52:43.551 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43538.log 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44718.log: /var/log/ceph/ceph-client.admin.48969.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48969.log.gz 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44718.log.gz 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43146.log 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42858.log 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43538.log: /var/log/ceph/ceph-client.admin.43146.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43538.log.gz 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43146.log.gz 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40754.log 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47334.log 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42858.log: /var/log/ceph/ceph-client.admin.40754.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42858.log.gz 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40754.log.gz 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46830.log 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53505.log 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47334.log: /var/log/ceph/ceph-client.admin.46830.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47334.log.gz 2024-04-26T19:52:43.552 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46830.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48285.log 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50492.log 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53505.log: /var/log/ceph/ceph-client.admin.48285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47697.log 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53505.log.gz 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50492.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48285.log.gz 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50492.log.gz 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41712.log 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47082.log 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47697.log.gz 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41712.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.41712.log.gz 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50954.log 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49053.log 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47082.log.gz 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51604.log 2024-04-26T19:52:43.553 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49053.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.50954.log.gz 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47040.log 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49053.log.gz 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44928.log 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51604.log: /var/log/ceph/ceph-client.admin.47040.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47040.log.gz 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51604.log.gz 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47943.log 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51919.log 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44928.log.gz 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51961.log 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51919.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47943.log.gz 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51919.log.gz 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46662.log 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50366.log 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46662.log: /var/log/ceph/ceph-client.admin.51961.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51961.log.gz 2024-04-26T19:52:43.554 INFO:teuthology.orchestra.run.smithi079.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46662.log.gz 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48363.log 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40850.log 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50366.log: /var/log/ceph/ceph-client.admin.48363.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50366.log.gz 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48363.log.gz 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45054.log 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53253.log 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.45054.log: /var/log/ceph/ceph-client.admin.40850.log: 89.4% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40850.log.gz 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45054.log.gz 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43868.log 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34885.log 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53253.log: /var/log/ceph/ceph-client.admin.43868.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53253.log.gz 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43868.log.gz 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53379.log 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49551.log 2024-04-26T19:52:43.555 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34885.log: /var/log/ceph/ceph-client.admin.53379.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34885.log.gz 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53379.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53547.log 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49263.log 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49551.log: /var/log/ceph/ceph-client.admin.53547.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53547.log.gz 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49551.log.gz 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46578.log 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44760.log 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49263.log.gz 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41418.log 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44760.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46578.log.gz 89.7% -- replaced with /var/log/ceph/ceph-client.admin.44760.log.gz 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42464.log 2024-04-26T19:52:43.556 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46746.log 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41418.log.gz 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45759.log 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46746.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42464.log.gz 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46746.log.gz 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44448.log 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.45759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49137.log 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45759.log.gz 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41100.log 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49137.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44448.log.gz 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49137.log.gz 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44207.log 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46956.log 2024-04-26T19:52:43.557 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41100.log.gz 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53043.log 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46956.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44207.log.gz 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46956.log.gz 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48573.log 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49761.log 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53043.log.gz 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47985.log 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49761.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48573.log.gz 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49761.log.gz 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49343.log 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47901.log 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47985.log.gz 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43356.log 2024-04-26T19:52:43.558 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47901.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.49343.log.gz 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47901.log.gz 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48201.log 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43356.log: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53817.log 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43356.log.gz 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48741.log 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48201.log.gz 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51741.log 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53817.log.gz 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50324.log 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48741.log.gz/var/log/ceph/ceph-client.admin.51741.log: 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43104.log 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51741.log.gz 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49179.log 2024-04-26T19:52:43.559 INFO:teuthology.orchestra.run.smithi079.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50324.log.gz 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46536.log 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43104.log.gz 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50198.log 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49179.log.gz 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46067.log 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46536.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50198.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46536.log.gz 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.2.46067.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.46067.log.gz 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50198.log.gz 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51338.log 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48843.log 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51295.log 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51338.log.gz 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48843.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48843.log.gz 2024-04-26T19:52:43.560 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51688.log 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42978.log 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51295.log: /var/log/ceph/ceph-client.admin.51688.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51295.log.gz 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51688.log.gz 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53631.log 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42978.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42978.log.gz 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41332.log 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44209.log 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53631.log: /var/log/ceph/ceph-client.admin.41332.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53631.log.gz 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41332.log.gz 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42936.log 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44209.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44209.log.gz 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44201.log 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46284.log 2024-04-26T19:52:43.561 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.42936.log: /var/log/ceph/ceph-client.admin.44201.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42936.log.gz 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44201.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52959.log 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.46284.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46284.log.gz 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48447.log 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47775.log 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52959.log: /var/log/ceph/ceph-client.admin.48447.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52959.log.gz 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48447.log.gz 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52235.log 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47775.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47775.log.gz 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53001.log 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44210.log 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52235.log: /var/log/ceph/ceph-client.admin.53001.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52235.log.gz 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.53001.log.gz 2024-04-26T19:52:43.562 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34990.log 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.44210.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44210.log.gz 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44970.log 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41628.log 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.34990.log: /var/log/ceph/ceph-client.admin.44970.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34990.log.gz 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44970.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51646.log 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.41628.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41628.log.gz 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44886.log 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47487.log 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51646.log: /var/log/ceph/ceph-client.admin.44886.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51646.log.gz 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44886.log.gz 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49011.log 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47487.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47487.log.gz 2024-04-26T19:52:43.563 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42506.log 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53337.log 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.49011.log: /var/log/ceph/ceph-client.admin.42506.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49011.log.gz 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42506.log.gz 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52685.log 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.53337.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53337.log.gz 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48243.log 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48783.log 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.52685.log: /var/log/ceph/ceph-client.admin.48243.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.52685.log.gz 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48243.log.gz 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28581.log 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48783.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48783.log.gz 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45096.log 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50642.log 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph.tmp-client.admin.28581.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28581.log.gz 2024-04-26T19:52:43.564 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.45096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48885.log 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45096.log.gz 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50642.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50642.log.gz 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50576.log 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46200.log 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.48885.log: /var/log/ceph/ceph-client.admin.50576.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48885.log.gz 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47817.log 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr: 91.0%/var/log/ceph/ceph-client.admin.46200.log: -- replaced with /var/log/ceph/ceph-client.admin.50576.log.gz 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46200.log.gz 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46368.log 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40931.log 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.47817.log: /var/log/ceph/ceph-client.admin.46368.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47817.log.gz 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46368.log.gz 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43314.log 2024-04-26T19:52:43.565 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.40931.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40931.log.gz 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51121.log 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.43314.log: /var/log/ceph/ceph-client.admin.44206.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43314.log.gz 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50114.log 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.51121.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51121.log.gz 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48075.log 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50068.log 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50114.log: /var/log/ceph/ceph-client.admin.48075.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50114.log.gz 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48075.log.gz 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr:/var/log/ceph/ceph-client.admin.50068.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50068.log.gz 2024-04-26T19:52:43.566 INFO:teuthology.orchestra.run.smithi079.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-26T19:52:43.711 INFO:teuthology.orchestra.run.smithi079.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T19:52:44.008 INFO:teuthology.orchestra.run.smithi083.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-26T19:52:44.055 INFO:teuthology.orchestra.run.smithi083.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-26T19:52:44.066 INFO:teuthology.orchestra.run.smithi083.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-26T19:52:44.067 INFO:teuthology.orchestra.run.smithi079.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T19:52:44.069 INFO:teuthology.orchestra.run.smithi079.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-26T19:52:44.124 INFO:teuthology.orchestra.run.smithi079.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T19:52:44.133 INFO:teuthology.orchestra.run.smithi079.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T19:52:44.135 INFO:teuthology.orchestra.run.smithi079.stderr: 2024-04-26T19:52:44.135 INFO:teuthology.orchestra.run.smithi079.stderr:real 0m0.755s 2024-04-26T19:52:44.135 INFO:teuthology.orchestra.run.smithi079.stderr:user 0m3.149s 2024-04-26T19:52:44.136 INFO:teuthology.orchestra.run.smithi079.stderr:sys 0m0.259s 2024-04-26T19:52:44.138 INFO:teuthology.orchestra.run.smithi083.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-26T19:52:44.140 INFO:teuthology.orchestra.run.smithi083.stderr: 2024-04-26T19:52:44.140 INFO:teuthology.orchestra.run.smithi083.stderr:real 0m0.738s 2024-04-26T19:52:44.140 INFO:teuthology.orchestra.run.smithi083.stderr:user 0m2.778s 2024-04-26T19:52:44.140 INFO:teuthology.orchestra.run.smithi083.stderr:sys 0m0.114s 2024-04-26T19:52:44.140 INFO:tasks.ceph:Archiving logs... 2024-04-26T19:52:44.141 DEBUG:teuthology.misc:Transferring archived files from smithi079:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/remote/smithi079/log 2024-04-26T19:52:44.142 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T19:52:44.840 DEBUG:teuthology.misc:Transferring archived files from smithi083:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/remote/smithi083/log 2024-04-26T19:52:44.842 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T19:52:45.309 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T19:52:45.319 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T19:52:45.320 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T19:52:45.323 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T19:52:45.400 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T19:52:45.411 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T19:52:45.411 DEBUG:teuthology.orchestra.run.smithi079:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T19:52:45.414 DEBUG:teuthology.orchestra.run.smithi083:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T19:52:45.427 INFO:teuthology.orchestra.run.smithi079.stderr:bash: line 1: ntpq: command not found 2024-04-26T19:52:45.429 INFO:teuthology.orchestra.run.smithi083.stderr:bash: line 1: ntpq: command not found 2024-04-26T19:52:45.432 INFO:teuthology.orchestra.run.smithi079.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T19:52:45.432 INFO:teuthology.orchestra.run.smithi079.stdout:=============================================================================== 2024-04-26T19:52:45.432 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 13 +1528us[+1528us] +/- 41ms 2024-04-26T19:52:45.432 INFO:teuthology.orchestra.run.smithi079.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -783us[ -773us] +/- 73ms 2024-04-26T19:52:45.432 INFO:teuthology.orchestra.run.smithi079.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -490us[ -490us] +/- 80ms 2024-04-26T19:52:45.432 INFO:teuthology.orchestra.run.smithi079.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:52:45.433 INFO:teuthology.orchestra.run.smithi083.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T19:52:45.433 INFO:teuthology.orchestra.run.smithi083.stdout:=============================================================================== 2024-04-26T19:52:45.433 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 15 +1165us[+1532us] +/- 41ms 2024-04-26T19:52:45.433 INFO:teuthology.orchestra.run.smithi083.stdout:^* hv02.front.sepia.ceph.com 3 6 377 14 -1157us[ -790us] +/- 73ms 2024-04-26T19:52:45.433 INFO:teuthology.orchestra.run.smithi083.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -858us[ -491us] +/- 80ms 2024-04-26T19:52:45.433 INFO:teuthology.orchestra.run.smithi083.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T19:52:45.434 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T19:52:45.444 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T19:52:45.445 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T19:52:45.456 DEBUG:teuthology.orchestra.run.smithi079:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T19:52:45.542 DEBUG:teuthology.orchestra.run.smithi083:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T19:52:45.605 DEBUG:teuthology.orchestra.run.smithi079:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T19:52:45.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:52:45.634 DEBUG:teuthology.orchestra.run.smithi083:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T19:52:45.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:52:45.662 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T19:52:45.672 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T19:52:45.685 INFO:teuthology.task.internal:Duration was 552.827427 seconds 2024-04-26T19:52:45.685 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T19:52:45.698 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T19:52:45.698 DEBUG:teuthology.orchestra.run.smithi079:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T19:52:45.701 DEBUG:teuthology.orchestra.run.smithi083:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T19:52:45.738 INFO:teuthology.orchestra.run.smithi079.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T19:52:45.739 INFO:teuthology.orchestra.run.smithi083.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T19:52:46.115 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T19:52:46.115 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi079.front.sepia.ceph.com 2024-04-26T19:52:46.116 DEBUG:teuthology.orchestra.run.smithi079:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T19:52:46.175 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi083.front.sepia.ceph.com 2024-04-26T19:52:46.175 DEBUG:teuthology.orchestra.run.smithi083:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T19:52:46.195 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T19:52:46.196 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T19:52:46.218 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T19:52:46.261 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T19:52:46.261 DEBUG:teuthology.orchestra.run.smithi079:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T19:52:46.285 DEBUG:teuthology.orchestra.run.smithi083:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T19:52:46.398 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T19:52:46.408 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T19:52:46.409 DEBUG:teuthology.orchestra.run.smithi079:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T19:52:46.434 DEBUG:teuthology.orchestra.run.smithi083:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T19:52:46.457 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T19:52:46.468 DEBUG:teuthology.orchestra.run.smithi079:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T19:52:46.477 DEBUG:teuthology.orchestra.run.smithi083:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T19:52:46.496 INFO:teuthology.orchestra.run.smithi079.stdout:kernel.core_pattern = core 2024-04-26T19:52:46.521 INFO:teuthology.orchestra.run.smithi083.stdout:kernel.core_pattern = core 2024-04-26T19:52:46.533 DEBUG:teuthology.orchestra.run.smithi079:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T19:52:46.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:52:46.569 DEBUG:teuthology.orchestra.run.smithi083:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T19:52:46.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T19:52:46.587 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T19:52:46.599 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T19:52:46.599 DEBUG:teuthology.misc:Transferring archived files from smithi079:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/remote/smithi079 2024-04-26T19:52:46.600 DEBUG:teuthology.orchestra.run.smithi079:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T19:52:46.642 DEBUG:teuthology.misc:Transferring archived files from smithi083:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-26_19:30:57-fs-wip-rishabh-testing-20240426.111959-testing-default-smithi/7675111/remote/smithi083 2024-04-26T19:52:46.642 DEBUG:teuthology.orchestra.run.smithi083:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T19:52:46.675 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T19:52:46.675 DEBUG:teuthology.orchestra.run.smithi079:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T19:52:46.679 DEBUG:teuthology.orchestra.run.smithi083:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T19:52:46.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T19:52:46.736 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T19:52:46.736 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T19:52:46.751 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T19:52:46.752 DEBUG:teuthology.orchestra.run.smithi079:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T19:52:46.754 DEBUG:teuthology.orchestra.run.smithi083:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T19:52:46.767 INFO:teuthology.orchestra.run.smithi079.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 19:52 /home/ubuntu/cephtest 2024-04-26T19:52:46.780 INFO:teuthology.orchestra.run.smithi083.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 19:52 /home/ubuntu/cephtest 2024-04-26T19:52:46.781 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T19:52:46.795 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T19:52:46.829 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/snap_schedule_snapdir} duration: 552.8274269104004 flavor: default owner: scheduled_rishabh@teuthology success: true 2024-04-26T19:52:46.829 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T19:52:46.909 INFO:teuthology.run:pass